05:28:07,261 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 05:28:07,768 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2703965530412460806] 05:28:07,799 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:07,959 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:07,959 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:08,52 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:08,52 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:08,362 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:08,363 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:08,381 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:08,394 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:08,412 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build what is described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#0.3 WARNING: A Java agent has been loaded dynamically (/tmp/r/net/bytebuddy/byte-buddy-agent/1.15.3/byte-buddy-agent-1.15.3.jar) WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information WARNING: Dynamic loading of agents will be disallowed by default in a future release Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build what is described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#0.3 05:28:09,704 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 05:28:09,876 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:09,895 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:09,970 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:09,975 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 05:28:09,997 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 05:28:10,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 05:28:10,45 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:10,45 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 05:28:10,45 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 05:28:10,98 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c63050a 05:28:10,108 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 05:28:10,128 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:10,129 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 05:28:10,131 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 05:28:10,131 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 05:28:10,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#773515531] 05:28:10,133 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 05:28:10,134 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 05:28:10,134 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 05:28:10,135 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#495795383] 05:28:10,136 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#773515531] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,144 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#495795383] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,157 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testBackupDatastore, persistent : true 05:28:10,157 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testBackupDatastore, persistent : true 05:28:10,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,175 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,176 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 05:28:10,227 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,228 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,228 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 05:28:10,279 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,280 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,280 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 05:28:10,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-1831840061] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,326 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1260837198 created and ready for shard:member-1-shard-people-config_testBackupDatastore 05:28:10,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1260837198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,327 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-1831840061 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 05:28:10,330 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,331 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,331 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 05:28:10,338 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1913836552] 05:28:10,339 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-1913836552] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,343 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#353679976] 05:28:10,343 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testBackupDatastore with journal batch size 1 05:28:10,343 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#353679976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,346 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testBackupDatastore with journal batch size 1 05:28:10,364 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:10,365 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:10,373 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:10,373 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:10,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:10,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 05:28:10,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:10,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 05:28:10,381 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,381 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,382 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#773515531] 05:28:10,394 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,394 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,395 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 05:28:10,395 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 05:28:10,396 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 05:28:10,397 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 05:28:10,413 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 05:28:10,413 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 05:28:10,417 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 05:28:10,417 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 05:28:10,418 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-config_testBackupDatastore 05:28:10,418 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-config_testBackupDatastore 05:28:10,423 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 05:28:10,423 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 05:28:10,423 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:10,423 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:10,424 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,424 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,424 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 05:28:10,424 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 05:28:10,424 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 05:28:10,424 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 05:28:10,468 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Received ElectionTimeout 05:28:10,472 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Received ElectionTimeout 05:28:10,477 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 05:28:10,477 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 05:28:10,477 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:10,477 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:10,477 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 05:28:10,477 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 05:28:10,479 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 05:28:10,479 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 05:28:10,480 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 05:28:10,480 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 05:28:10,488 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 05:28:10,489 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-config_testBackupDatastore 05:28:10,489 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testBackupDatastore, leaderId=member-1-shard-cars-config_testBackupDatastore, leaderPayloadVersion=13] 05:28:10,493 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 05:28:10,493 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testBackupDatastore, leaderId=member-1-shard-people-config_testBackupDatastore, leaderPayloadVersion=13] 05:28:10,493 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 05:28:10,493 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 05:28:10,584 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,585 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:28:10,585 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#495795383] 05:28:10,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:10,598 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:10,608 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 05:28:10,611 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#245489623] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,612 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:10,616 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:10,616 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 05:28:10,616 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 05:28:10,617 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 05:28:10,617 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:10,617 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 05:28:10,617 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 05:28:10,618 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@8ed28b7 05:28:10,618 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 05:28:10,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:10,619 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 05:28:10,621 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-cars-oper_testBackupDatastore, persistent : true 05:28:10,622 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testBackupDatastore with journal batch size 1 05:28:10,622 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#2077148669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,622 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/shard-member-1:snapshot-read#731155567] 05:28:10,626 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 05:28:10,626 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 05:28:10,626 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#2077148669] 05:28:10,626 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 05:28:10,627 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 05:28:10,627 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 05:28:10,627 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#207969608] 05:28:10,629 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testBackupDatastore, persistent : true 05:28:10,630 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1208087030 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 05:28:10,630 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-1063155833 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 05:28:10,631 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testBackupDatastore with journal batch size 1 05:28:10,631 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/shard-member-1:snapshot-read#2127067121] 05:28:10,632 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#1208087030] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:10,633 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:10,633 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:10,635 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,635 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:10,635 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 05:28:10,635 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 05:28:10,635 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 05:28:10,635 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 05:28:10,636 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-oper_testBackupDatastore 05:28:10,636 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 05:28:10,636 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:10,636 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,636 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 05:28:10,637 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 05:28:10,637 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 05:28:10,637 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Received ElectionTimeout 05:28:10,637 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 05:28:10,637 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:10,637 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 05:28:10,638 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 05:28:10,638 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 05:28:10,638 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 05:28:10,635 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:10,638 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:10,638 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,638 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testBackupDatastore 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 05:28:10,639 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 05:28:10,640 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 05:28:10,640 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 05:28:10,640 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testBackupDatastore, leaderId=member-1-shard-people-oper_testBackupDatastore, leaderPayloadVersion=13] 05:28:10,640 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 05:28:10,640 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:10,640 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 05:28:10,641 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Received ElectionTimeout 05:28:10,641 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 05:28:10,641 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:10,641 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 05:28:10,641 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 05:28:10,641 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 05:28:10,641 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener akka://cluster-test/user/shardmanager-oper_testBackupDatastore 05:28:10,642 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testBackupDatastore, leaderId=member-1-shard-cars-oper_testBackupDatastore, leaderPayloadVersion=13] 05:28:10,642 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 05:28:10,642 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 05:28:10,692 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#245489623] 05:28:10,693 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:10,693 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#2077148669] 05:28:10,744 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore#245489623] 05:28:10,744 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:28:10,745 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#207969608] 05:28:10,766 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 05:28:10,767 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 05:28:10,771 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 05:28:10,771 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onGetSnapshot 05:28:10,772 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,772 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 05:28:10,773 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,783 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 05:28:10,784 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 05:28:10,784 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 05:28:10,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 05:28:10,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,789 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@63dcced, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,789 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4533a0f8, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,807 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@210dfa35, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,808 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 05:28:10,808 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7bd34baa, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,808 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 05:28:10,849 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 05:28:10,881 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore#60127815] 05:28:10,881 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@7f0c9e34 05:28:10,882 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:10,883 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#773515531] 05:28:10,919 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testBackupDatastore 05:28:10,919 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Aborting 0 pending queued transactions 05:28:10,925 PM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 05:28:10,926 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 05:28:10,926 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 05:28:10,928 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 05:28:10,928 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 05:28:10,928 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 05:28:10,928 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,928 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 05:28:10,928 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,929 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,929 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,929 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@10c510ea, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,929 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@38648979, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 05:28:10,929 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 05:28:10,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 05:28:10,930 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:10,931 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:10,931 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@f481d2a, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 05:28:11,151 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 05:28:11,153 PM [cluster-test-akka.actor.default-dispatcher-17] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:332) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 05:28:11,165 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 05:28:11,168 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 05:28:11,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 05:28:11,170 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:28:11,172 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onShutDown 05:28:11,174 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@49bd494 05:28:11,174 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@49bd494 05:28:11,175 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-people-config_testBackupDatastore 05:28:11,175 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Aborting 0 pending queued transactions 05:28:11,176 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:11,178 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:11,180 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 05:28:11,183 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 05:28:11,183 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 05:28:11,183 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onShutDown 05:28:11,183 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:11,184 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6e89eb5 05:28:11,184 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6e89eb5 05:28:11,184 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onShutDown 05:28:11,184 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@49a2e56f 05:28:11,184 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@49a2e56f 05:28:11,184 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testBackupDatastore 05:28:11,184 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Aborting 0 pending queued transactions 05:28:11,185 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testBackupDatastore 05:28:11,185 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Aborting 0 pending queued transactions 05:28:11,185 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:11,185 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:11,194 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:11,195 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:11,226 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:11,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:11,306 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5250687189719934976] 05:28:11,308 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:11,319 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:11,320 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:11,323 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:11,323 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:11,364 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:11,365 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:11,365 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:11,369 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,375 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,382 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 05:28:11,392 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#702372597] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,396 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:11,404 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:11,404 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,405 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 05:28:11,406 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:11,406 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,407 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@72522955 05:28:11,410 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 05:28:11,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:11,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,414 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,414 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,414 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227] 05:28:11,414 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,415 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,415 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,415 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811] 05:28:11,416 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddReplicasForAllShards, persistent : true 05:28:11,418 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 05:28:11,418 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testAddReplicasForAllShards, persistent : true 05:28:11,418 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,419 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,419 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-2026475098] 05:28:11,420 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testAddReplicasForAllShards with journal batch size 1 05:28:11,420 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#1930766108 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1897870965 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,421 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1958330568] 05:28:11,421 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-2026475098] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,423 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#1930766108] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,423 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1897870965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,423 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-1958330568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,425 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:11,425 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:11,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:11,426 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:11,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 05:28:11,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testAddReplicasForAllShards 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:11,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 05:28:11,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:11,428 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 05:28:11,428 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:11,428 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 05:28:11,428 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,428 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 05:28:11,429 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 05:28:11,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddReplicasForAllShards 05:28:11,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 05:28:11,429 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:11,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,429 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 05:28:11,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 05:28:11,429 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 05:28:11,430 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 05:28:11,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 05:28:11,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 05:28:11,430 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:11,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:11,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 05:28:11,430 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 05:28:11,431 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:11,431 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:11,431 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:11,431 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 05:28:11,431 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:11,432 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 05:28:11,432 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:11,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#702372597] 05:28:11,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:11,463 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227] 05:28:11,513 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#702372597] 05:28:11,514 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:28:11,514 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811] 05:28:11,514 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,521 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,527 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 05:28:11,529 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1866350078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,532 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:11,534 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:11,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,534 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,535 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:11,535 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,536 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,537 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 05:28:11,538 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f121e9a 05:28:11,538 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 05:28:11,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:11,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,540 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddReplicasForAllShards, persistent : true 05:28:11,541 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 05:28:11,542 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,542 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#-72310854] 05:28:11,542 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1966060754 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,542 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1966060754] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,543 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,543 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,544 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:11,544 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:11,544 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,544 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 05:28:11,545 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,545 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 05:28:11,545 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 05:28:11,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693] 05:28:11,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 05:28:11,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 05:28:11,547 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:11,547 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:11,547 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 05:28:11,547 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 05:28:11,547 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#268456539] 05:28:11,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:11,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 05:28:11,548 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testAddReplicasForAllShards, persistent : true 05:28:11,549 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#1961722392 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,549 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 05:28:11,550 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:11,550 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 05:28:11,550 PM [cluster-test-akka.actor.default-dispatcher-7] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#1143594500] 05:28:11,551 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 05:28:11,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:11,552 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:11,552 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:11,552 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 05:28:11,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 05:28:11,553 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testAddReplicasForAllShards 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 05:28:11,553 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 05:28:11,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 05:28:11,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 05:28:11,554 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:11,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:11,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 05:28:11,554 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 05:28:11,555 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 05:28:11,555 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:11,555 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 05:28:11,555 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 05:28:11,590 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1866350078] 05:28:11,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:11,591 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693] 05:28:11,642 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1866350078] 05:28:11,642 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:28:11,643 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#268456539] 05:28:11,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3c51267] 05:28:11,665 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 05:28:11,667 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-pets-config_testAddReplicasForAllShards, persistent : true 05:28:11,668 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 05:28:11,669 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#28913744] 05:28:11,669 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-645839763 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 05:28:11,671 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:11,671 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:11,671 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 05:28:11,672 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 05:28:11,672 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-pets-config_testAddReplicasForAllShards 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:11,672 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 05:28:11,672 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 05:28:11,673 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 05:28:11,673 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 05:28:11,673 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:11,673 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:11,673 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 05:28:11,674 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 05:28:11,674 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:11,674 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:11,674 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 05:28:11,674 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:11,721 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#702372597] 05:28:11,722 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 05:28:11,722 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-113603506] 05:28:11,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:11,777 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1726113274548103912] 05:28:11,777 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:11,784 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:11,784 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:11,784 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:11,785 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:11,818 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,823 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,835 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 05:28:11,856 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1967934212] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,856 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:11,865 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:11,869 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 05:28:11,870 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@584e84ac 05:28:11,870 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 05:28:11,871 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:11,871 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,875 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:11,879 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 05:28:11,880 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#857111364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:11,881 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:11,883 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:11,884 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 05:28:11,885 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52b6efdc 05:28:11,887 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 05:28:11,888 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:12,163 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:12,218 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:12,218 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:12,218 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,218 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:12,218 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 05:28:12,340 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:12,343 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:12,343 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,344 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:12,344 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,344 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:12,344 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 05:28:12,344 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,345 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,406 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3b73e650] 05:28:12,408 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 05:28:12,410 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-pets-config_testAddReplicasForAllShards, persistent : true 05:28:12,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@53fead64] 05:28:12,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 05:28:12,411 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 05:28:12,412 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-no-leader-oper_testAddReplicasForAllShards, persistent : true 05:28:12,413 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-no-leader-oper_testAddReplicasForAllShards with journal batch size 1 05:28:12,413 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1219600463 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,414 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1427060526 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 05:28:12,414 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#436315679] 05:28:12,414 PM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 05:28:12,414 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1355024518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,414 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 05:28:12,415 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:12,415 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 05:28:12,415 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:12,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 05:28:12,418 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-166853631] 05:28:12,418 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 05:28:12,418 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,418 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is no longer the leader 05:28:12,418 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,418 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 05:28:12,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:12,419 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 05:28:12,419 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:12,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 05:28:12,421 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 05:28:12,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 05:28:12,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 05:28:12,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 05:28:12,426 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 05:28:12,427 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 05:28:12,428 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 05:28:12,428 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:12,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:12,431 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,431 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,431 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,431 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 05:28:12,431 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,431 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 05:28:12,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 05:28:12,430 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227, primaryVersion=13] 05:28:12,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 05:28:12,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 05:28:12,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 05:28:12,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693, primaryVersion=13] 05:28:12,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 05:28:12,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 05:28:12,434 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:12,435 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811, primaryVersion=13] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-113603506, primaryVersion=13] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#268456539, primaryVersion=13] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 05:28:12,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 05:28:12,444 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227 for shard member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,450 PM [cluster-test-shard-dispatcher-12] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 05:28:12,450 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 05:28:12,450 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 05:28:12,451 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddReplicasForAllShards, persistent : true 05:28:12,452 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-533439179] 05:28:12,452 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-1568981983 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,452 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 05:28:12,453 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddReplicasForAllShards, persistent : true 05:28:12,453 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#85301660 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,453 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1191770483] 05:28:12,453 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 05:28:12,456 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,457 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,457 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 05:28:12,457 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,457 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,457 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-373169619] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1219600463] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1427060526] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#436315679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-166853631] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#376375919] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,458 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-1568981983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:12,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 05:28:12,461 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ClusterAdminRpcService - Failed to add replica for shard no-leader, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1051) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:207) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 05:28:12,458 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 05:28:12,464 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,465 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 05:28:12,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,467 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 05:28:12,468 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811 for shard member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,469 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#268456539 for shard member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,469 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:12,469 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 05:28:12,470 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-oper_testAddReplicasForAllShards, persistent : true 05:28:12,471 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 05:28:12,471 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testAddReplicasForAllShards, persistent : true 05:28:12,472 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testAddReplicasForAllShards with journal batch size 1 05:28:12,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 05:28:12,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,472 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:12,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 05:28:12,473 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#323333237] 05:28:12,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-133688511 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,473 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 05:28:12,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 05:28:12,473 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,470 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-113603506 for shard member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,473 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:12,473 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 05:28:12,473 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:12,473 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 05:28:12,474 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:12,474 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 05:28:12,476 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,476 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,477 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,477 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 05:28:12,477 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:12,477 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:28:12,473 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#1621400571] 05:28:12,473 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#840177381 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 05:28:12,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,478 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:12,478 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 05:28:12,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:12,479 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 05:28:12,479 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 05:28:12,479 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 05:28:12,480 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 05:28:12,480 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 05:28:12,478 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 05:28:12,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:12,477 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,482 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 05:28:12,482 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:12,477 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,483 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 05:28:12,483 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:12,477 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 05:28:12,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:12,475 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,484 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 05:28:12,484 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:12,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,485 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 05:28:12,485 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@283be7fc, electionTerm=1, electionVotedFor=member-1-shard-pets-config_testAddReplicasForAllShards, ServerConfigPayload=null] 05:28:12,485 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:12,486 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,486 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 05:28:12,486 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@23c68ca, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 05:28:12,486 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:12,486 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,486 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 05:28:12,486 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@78042f1a, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 05:28:12,486 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:12,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 05:28:12,486 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,488 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@e88bf59, electionTerm=1, electionVotedFor=member-1-shard-people-config_testAddReplicasForAllShards, ServerConfigPayload=null] 05:28:12,488 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 05:28:12,488 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:12,488 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@21ca2a49, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddReplicasForAllShards, ServerConfigPayload=null] 05:28:12,494 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:12,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 05:28:12,495 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 05:28:12,496 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddReplicasForAllShards: 229 05:28:12,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 05:28:12,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-pets-config_testAddReplicasForAllShards: 229 05:28:12,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 05:28:12,496 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-oper_testAddReplicasForAllShards: 229 05:28:12,495 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 05:28:12,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-config_testAddReplicasForAllShards: 229 05:28:12,497 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddReplicasForAllShards: 229 05:28:12,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-people-config_testAddReplicasForAllShards), Chunk: 1/1 05:28:12,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-cars-oper_testAddReplicasForAllShards), Chunk: 1/1 05:28:12,498 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,498 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-cars-config_testAddReplicasForAllShards), Chunk: 1/1 05:28:12,498 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,498 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,498 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,498 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,502 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-people-oper_testAddReplicasForAllShards), Chunk: 1/1 05:28:12,502 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,502 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,503 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,503 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,503 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,503 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 05:28:12,503 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a journal response 05:28:12,504 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,504 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 05:28:12,504 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 05:28:12,504 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,504 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,504 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a journal response 05:28:12,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-pets-config_testAddReplicasForAllShards), Chunk: 1/1 05:28:12,504 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,505 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,506 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,506 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a journal response 05:28:12,506 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a journal response 05:28:12,512 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:12,512 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:12,516 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,512 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:12,516 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:12,510 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:12,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,517 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:12,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,516 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,517 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:12,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:12,511 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:12,517 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,517 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:12,519 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:12,519 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:12,519 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,519 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:12,519 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-oper_testAddReplicasForAllShards 05:28:12,520 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:12,520 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:12,520 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:12,520 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,521 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,521 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,521 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 05:28:12,521 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 05:28:12,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:12,522 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 05:28:12,522 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@30c1bf48, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:12,522 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:12,522 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 05:28:12,522 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6b9290e0, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:12,523 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:12,523 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,523 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 05:28:12,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 05:28:12,523 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,523 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:12,523 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,523 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 05:28:12,523 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 05:28:12,524 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 05:28:12,524 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ff4e214, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:12,524 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:12,524 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:12,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,524 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 05:28:12,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:12,524 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7c0889d2, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:12,524 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:12,525 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,525 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 05:28:12,525 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:12,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,525 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 05:28:12,525 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,525 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 05:28:12,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 05:28:12,525 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,525 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,526 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 05:28:12,526 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7d478d7, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:12,526 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 05:28:12,526 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 05:28:12,527 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 05:28:12,527 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:12,540 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#949537569] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 05:28:12,541 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-2096869343] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 05:28:12,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1314092129] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 05:28:12,540 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#199394113] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 05:28:12,541 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 14.52 ms 05:28:12,541 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 16.90 ms 05:28:12,542 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 05:28:12,542 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 05:28:12,541 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 15.52 ms 05:28:12,542 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 05:28:12,541 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 16.48 ms 05:28:12,543 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 05:28:12,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#949537569] 05:28:12,543 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,545 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1314092129] 05:28:12,545 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#128165803] 05:28:12,545 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 05:28:12,542 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,546 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,542 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,546 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,547 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a journal response 05:28:12,547 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 05:28:12,547 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a journal response 05:28:12,547 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,547 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,547 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 05:28:12,547 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 05:28:12,542 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,550 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 4.806 ms 05:28:12,550 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#199394113] 05:28:12,550 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,550 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:12,550 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 05:28:12,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:12,551 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 108.1 μs 05:28:12,545 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 05:28:12,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,551 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 6.257 ms 05:28:12,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddReplicasForAllShards] 05:28:12,551 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-2096869343] 05:28:12,552 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 05:28:12,552 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 167.8 μs 05:28:12,551 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,552 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:12,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:12,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-oper_testAddReplicasForAllShards] 05:28:12,551 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:12,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:12,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-config_testAddReplicasForAllShards] 05:28:12,552 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]] 05:28:12,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]] 05:28:12,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]] 05:28:12,554 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,554 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,555 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a journal response 05:28:12,555 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,555 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:12,556 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:12,556 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddReplicasForAllShards] 05:28:12,556 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]] 05:28:12,556 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#128165803] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 05:28:12,556 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,560 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,557 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,562 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=17cfb4b3-40d9-4c5a-8d62-820f9e631908}, logIndex: 0, isSendImmediate: true 05:28:12,557 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,556 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=654386ad-2282-4f05-b4f1-41d020f0e704}, logIndex: 0, isSendImmediate: true 05:28:12,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=468a6f8c-3ac9-433a-91e4-cc74fea03638}, logIndex: 0, isSendImmediate: true 05:28:12,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,563 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=de1acaf6-97ed-4a8e-8a91-73a8bd924882}, logIndex: 0, isSendImmediate: true 05:28:12,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,565 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,565 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,566 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,566 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,571 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 05:28:12,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,571 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 164.0 μs 05:28:12,565 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,572 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,576 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,576 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,566 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,577 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,564 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,578 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,581 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,581 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 05:28:12,581 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:12,582 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,582 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,582 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 05:28:12,582 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:12,582 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,582 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards, votingState=VOTING]] 05:28:12,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Log size is now 1 05:28:12,583 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#936237811 05:28:12,584 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 05:28:12,585 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,585 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 05:28:12,582 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,585 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,585 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards, votingState=VOTING]] 05:28:12,585 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Log size is now 1 05:28:12,586 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:28:12,586 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 05:28:12,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 59.43 ms 05:28:12,586 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 05:28:12,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,586 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:12,582 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#2066406693 05:28:12,587 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 05:28:12,587 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,587 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 05:28:12,587 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:28:12,587 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 05:28:12,585 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:12,588 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:12,588 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:12,589 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:12,589 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,589 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:12,589 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:12,589 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 05:28:12,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:12,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 05:28:12,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 05:28:12,590 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 05:28:12,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:12,590 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,590 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:12,590 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 05:28:12,591 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 05:28:12,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 05:28:12,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:12,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,595 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,595 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards, votingState=VOTING]] 05:28:12,595 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Log size is now 1 05:28:12,595 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:12,595 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,595 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 05:28:12,595 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:12,595 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:12,595 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-pets-config_testAddReplicasForAllShards] 05:28:12,595 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#268456539 05:28:12,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 05:28:12,595 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-390626227 05:28:12,596 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 05:28:12,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 05:28:12,596 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:28:12,595 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]] 05:28:12,596 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 05:28:12,596 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,596 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=79777b24-2d3e-4267-9389-d18c6b3ed3ff}, logIndex: 0, isSendImmediate: true 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 05:28:12,597 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:12,598 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,599 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,588 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 05:28:12,600 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a journal response 05:28:12,600 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 05:28:12,597 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars, people] 05:28:12,596 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,601 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 05:28:12,601 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:12,601 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,601 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:12,602 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testAddReplicasForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards, votingState=VOTING]] 05:28:12,602 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Log size is now 1 05:28:12,602 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:12,602 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 05:28:12,602 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:28:12,602 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:12,602 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:12,602 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:12,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:12,596 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,602 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 05:28:12,602 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 05:28:12,603 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 05:28:12,603 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:12,604 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 05:28:12,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:12,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 05:28:12,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,605 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:12,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 05:28:12,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:12,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-oper_testAddReplicasForAllShards 05:28:12,605 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,606 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 05:28:12,606 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 05:28:12,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testAddReplicasForAllShards, address=null, votingState=VOTING]] 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Log size is now 1 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:12,612 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: getLeaderAddress leaderId = member-1-shard-pets-config_testAddReplicasForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,613 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 05:28:12,614 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 05:28:12,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 05:28:12,614 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 05:28:12,615 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-113603506 05:28:12,615 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 05:28:12,615 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,615 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 05:28:12,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:12,615 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 05:28:12,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,615 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 05:28:12,616 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 05:28:12,616 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 05:28:12,616 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:12,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:12,617 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,618 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1967934212] 05:28:12,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 05:28:12,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:12,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 05:28:12,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,619 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,619 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 05:28:12,619 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:12,620 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#376375919] 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Commit index set to 0 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,621 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:12,621 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,621 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Commit index set to 0 05:28:12,622 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,622 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 05:28:12,623 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#376375919] 05:28:12,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Commit index set to 0 05:28:12,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:12,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Setting last applied to 0 05:28:12,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,624 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,626 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1967934212] 05:28:12,626 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,626 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 05:28:12,626 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:28:12,626 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1733235022] 05:28:12,626 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1733235022] 05:28:12,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1967934212] 05:28:12,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 05:28:12,627 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1355024518] 05:28:12,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-1355024518] 05:28:12,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#857111364] 05:28:12,628 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:12,628 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1948931624] 05:28:12,628 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1948931624] 05:28:12,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#857111364] 05:28:12,629 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:28:12,629 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#1142247344] 05:28:12,629 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#1142247344] 05:28:12,629 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 05:28:12,630 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,630 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,630 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onShutDown 05:28:12,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Commit index set to 0 05:28:12,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:12,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Setting last applied to 0 05:28:12,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 05:28:12,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,631 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddReplicasForAllShards 05:28:12,631 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,630 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 05:28:12,631 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onShutDown 05:28:12,631 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 05:28:12,632 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 05:28:12,632 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onShutDown 05:28:12,632 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testAddReplicasForAllShards 05:28:12,632 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,632 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 05:28:12,633 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-config_testAddReplicasForAllShards 05:28:12,633 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,633 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:12,633 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:12,635 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 05:28:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 05:28:12,635 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onShutDown 05:28:12,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 05:28:12,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onShutDown 05:28:12,636 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onShutDown 05:28:12,636 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddReplicasForAllShards 05:28:12,636 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 05:28:12,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,636 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,637 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testAddReplicasForAllShards 05:28:12,637 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:12,637 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:12,637 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:12,639 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:12,639 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:12,665 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:12,665 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:12,667 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:12,704 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 05:28:12,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 05:28:12,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 05:28:12,706 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 05:28:12,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onShutDown 05:28:12,706 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 05:28:12,707 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onShutDown 05:28:12,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating leader transfer 05:28:12,707 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating leader transfer 05:28:12,708 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onShutDown 05:28:12,708 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating leader transfer 05:28:12,709 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 05:28:12,710 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 05:28:12,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@211a66eb 05:28:12,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@211a66eb 05:28:12,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 05:28:12,710 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 05:28:12,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b9318 05:28:12,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b9318 05:28:12,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 05:28:12,711 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 05:28:12,711 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 05:28:12,711 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,711 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:12,712 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@301e0a8 05:28:12,712 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@301e0a8 05:28:12,712 PM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 05:28:12,712 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 05:28:12,713 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:13,664 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1726113274548103912] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:17,910 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:18,459 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 05:28:18,462 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:18,462 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:18,517 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:18,546 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:20,147 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:20,467 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:20,555 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:21,106 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:21,429 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:21,578 PM [cluster-test-akka.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:21,732 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 05:28:21,733 PM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.024 s 05:28:21,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 05:28:21,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 05:28:21,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 05:28:21,734 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,736 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddReplicasForAllShards] 05:28:21,736 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:21,736 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:28:21,737 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:21,737 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 05:28:21,737 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,738 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddReplicasForAllShards 05:28:21,738 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:21,743 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,743 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testAddReplicasForAllShards] 05:28:21,743 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:21,744 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:28:21,744 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:21,744 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 05:28:21,744 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 05:28:21,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 05:28:21,753 PM [cluster-test-shard-dispatcher-33] [WARN] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,753 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.043 s 05:28:21,753 PM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.044 s 05:28:21,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 05:28:21,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 05:28:21,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 05:28:21,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testAddReplicasForAllShards] 05:28:21,753 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 05:28:21,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 05:28:21,754 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 05:28:21,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:21,754 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,754 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:28:21,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:21,754 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testAddReplicasForAllShards] 05:28:21,754 PM [cluster-test-akka.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 05:28:21,754 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:21,754 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,755 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:28:21,755 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testAddReplicasForAllShards 05:28:21,755 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:28:21,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:21,755 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 05:28:21,755 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 05:28:21,755 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testAddReplicasForAllShards 05:28:21,756 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:21,756 PM [cluster-test-akka.actor.default-dispatcher-7] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:21,757 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:21,757 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 05:28:21,760 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:21,760 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 05:28:21,760 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:21,761 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onShutDown 05:28:21,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onShutDown 05:28:21,761 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testAddReplicasForAllShards 05:28:21,761 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddReplicasForAllShards 05:28:21,761 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:21,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Aborting 0 pending queued transactions 05:28:21,762 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:21,762 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:21,764 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:21,765 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:21,769 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:21.773] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:21,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:21,823 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4764200201422468602] 05:28:21,824 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:21,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:21,828 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:21,829 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:21,829 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:21,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:21,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:21,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:21,852 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:21,855 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:21,858 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 05:28:21,868 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:21,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,878 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:21,878 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,878 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,879 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:21,879 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,879 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,890 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:21,891 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 05:28:21,893 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:21,897 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:21,897 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b5f591b 05:28:21,897 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:21,897 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,901 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 05:28:21,903 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,903 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,903 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:21,903 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,904 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,904 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,904 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] 05:28:21,905 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStates, persistent : true 05:28:21,906 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,906 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#1184175507 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 05:28:21,906 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#1184175507] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,908 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,910 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:21,910 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:21,911 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,911 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,911 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:21,911 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,911 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,912 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStates, persistent : true 05:28:21,912 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStates with journal batch size 1 05:28:21,912 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#30100595] 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#30100595] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1089240012 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1089240012] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1262094738] 05:28:21,913 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#-1262094738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,914 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:21,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:21,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:21,922 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:21,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:21,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:21,923 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:21,923 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:21,923 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:21,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:21,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:21,923 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:21,923 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:21,924 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:21,924 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 05:28:21,929 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:21,929 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2c46ce4a 05:28:21,930 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 05:28:21,936 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:21,937 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,938 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStates, persistent : true 05:28:21,938 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,938 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,938 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:21,938 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,939 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,939 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 05:28:21,939 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,939 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1693380386] 05:28:21,940 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,940 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#655527282 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:21,940 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#821358213] 05:28:21,940 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStates, persistent : true 05:28:21,940 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#655527282] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:21,942 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 05:28:21,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#-156407274] 05:28:21,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:21,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:21,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:21,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 05:28:21,942 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 05:28:21,943 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:21,943 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:21,943 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:21,943 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 05:28:21,943 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:21,943 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:21,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:21,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:21,944 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 05:28:21,944 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:21,944 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:21,944 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:21,945 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:21,945 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:21,945 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#494420398 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 05:28:21,945 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:21,945 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:21,945 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 05:28:21,950 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:21,964 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6341693480696104090] 05:28:21,965 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:21,968 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:21,968 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:21,970 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:21,970 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:21,994 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,1 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,4 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 05:28:22,11 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1396835413] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,14 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:22,16 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:22,17 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,17 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,29 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:22,30 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7009fd02 05:28:22,30 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 05:28:22,31 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,33 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,34 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:22,34 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,37 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 05:28:22,41 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,42 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStates, persistent : true 05:28:22,43 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:22,43 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 05:28:22,43 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStates, persistent : true 05:28:22,44 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1766832072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,44 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1488854125] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,44 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1378628571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,44 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-520545516] 05:28:22,44 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStates with journal batch size 1 05:28:22,44 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-824015187] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,45 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#217056943] 05:28:22,46 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-668794501 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,47 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-520545516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,47 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#217056943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,47 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-668794501] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:22,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,57 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,50 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#-824015187 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,57 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:22,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,57 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 05:28:22,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:22,57 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,48 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:22,57 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:22,57 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 05:28:22,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,58 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,59 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,59 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:22,59 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:28:22,59 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 05:28:22,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:22,59 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 05:28:22,76 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:22,78 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 05:28:22,79 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@59390d7 05:28:22,80 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:22,81 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:22,81 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,82 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,83 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStates, persistent : true 05:28:22,83 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#2125603897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,83 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#1524916965] 05:28:22,83 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#2070842427 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,83 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 05:28:22,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#2070842427] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,84 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStates, persistent : true 05:28:22,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#343630547 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,84 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 05:28:22,84 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#1106994635] 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:22,85 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,85 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:22,86 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 05:28:22,86 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:22,86 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 05:28:22,89 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,89 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,89 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,89 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:22,89 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:28:22,90 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 05:28:22,90 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,90 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:22,90 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 05:28:22,94 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,95 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,98 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:22,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3875736336021850963] 05:28:22,112 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:28:22,116 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:22,117 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:28:22,118 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:22,118 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:22,141 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,145 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,148 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 05:28:22,158 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-403798589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,159 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:28:22,162 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:28:22,162 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,163 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,170 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:22,174 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b07e3e8 05:28:22,174 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 05:28:22,175 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,176 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:22,176 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,177 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,177 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStates, persistent : true 05:28:22,178 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 05:28:22,179 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStates, persistent : true 05:28:22,179 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2142905458] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,183 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:28:22,183 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:22,186 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 05:28:22,193 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-585156819 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,194 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1843713479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,194 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:22,194 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1253567976 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,194 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-805157714] 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStates with journal batch size 1 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-510093668] 05:28:22,195 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-585156819] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#1253567976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-805157714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-510093668] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,199 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1821394753] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,199 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:28:22,201 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,201 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,203 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:28:22,203 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,203 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,203 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:22,204 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,204 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:22,204 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,204 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,204 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,204 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,204 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:22,205 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 05:28:22,205 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:28:22,206 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 05:28:22,206 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:28:22,206 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:22,207 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 05:28:22,210 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:22,210 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 05:28:22,210 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 05:28:22,210 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 05:28:22,210 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,211 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:22,211 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,211 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,211 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@19ab6e19 05:28:22,211 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:22,211 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,212 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] 05:28:22,213 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStates, persistent : true 05:28:22,213 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,213 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:22,214 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#-349651045 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,214 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#-1152471664] 05:28:22,214 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStates, persistent : true 05:28:22,214 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,216 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:28:22,216 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 05:28:22,217 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#1127976148 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,217 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#502559821] 05:28:22,216 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,217 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:22,217 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,217 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,217 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 05:28:22,217 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:22,218 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:22,221 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:28:22,221 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:22,221 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 05:28:22,221 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,222 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 05:28:22,222 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 05:28:22,840 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:28:22,841 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,841 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:22,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,842 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,843 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,843 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:22,861 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:22,862 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,862 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,980 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:22,980 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 05:28:22,980 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 05:28:22,980 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,980 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 05:28:22,981 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:22,985 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:28:22,986 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,986 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:22,986 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:22,986 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:22,988 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:22,989 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 05:28:22,989 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:22,989 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 05:28:22,989 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStates 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,989 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:22,990 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:22,990 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 05:28:22,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStates 05:28:22,990 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:28:22,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:22,990 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:22,990 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:22,990 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:22,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,0 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:22,999 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,5 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,5 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:28:23,10 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:28:23,10 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:28:23,10 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:28:23,10 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:28:23,10 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:28:23,11 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:28:23,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,11 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,11 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,12 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,12 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,11 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:28:23,12 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:28:23,12 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:23,12 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,12 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:28:23,13 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,13 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,13 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:28:23,13 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,13 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 05:28:23,13 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,13 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:28:23,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,13 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:23,14 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,15 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,15 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:28:23,15 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,15 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,16 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 05:28:23,16 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,16 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,16 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 05:28:23,16 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,16 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,16 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,16 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,17 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,17 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,17 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,17 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,17 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 05:28:23,16 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 05:28:23,17 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,17 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,17 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,17 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,18 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,18 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,18 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,18 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,18 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,18 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,18 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,18 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,18 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,18 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:28:23,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:28:23,19 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:28:23,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 05:28:23,19 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,20 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,20 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,20 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,20 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:28:23,20 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,20 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,22 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 05:28:23,22 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 05:28:23,23 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,23 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,23 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,23 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,23 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,24 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,24 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,24 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,24 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,24 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,24 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,24 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,24 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,24 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 05:28:23,24 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,24 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,24 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 05:28:23,25 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,25 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,27 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,27 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,27 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,27 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,27 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,28 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,29 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,29 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,29 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,30 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,30 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,30 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,30 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,30 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,31 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,31 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,32 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,33 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:28:23,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,34 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 05:28:23,34 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,34 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 05:28:23,34 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 05:28:23,35 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,35 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,35 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,35 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,35 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 05:28:23,35 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,35 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:28:23,35 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,36 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,36 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,36 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 05:28:23,36 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,36 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,36 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,37 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,37 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,39 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,39 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,39 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,39 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,39 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,40 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,40 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:28:23,41 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,41 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,42 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,43 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,45 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,48 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-2-shard-people-oper_testFlipMemberVotingStates] 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:28:23,50 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,51 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 05:28:23,51 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 05:28:23,51 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,51 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 05:28:23,51 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:23,51 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 05:28:23,51 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:23,51 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 05:28:23,51 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,51 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:23,52 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,56 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,56 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,56 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,56 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 05:28:23,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:28:23,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 05:28:23,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 05:28:23,56 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 05:28:23,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 05:28:23,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:28:23,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,57 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 05:28:23,59 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 05:28:23,59 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 05:28:23,60 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 05:28:23,60 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:28:23,60 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:28:23,62 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:28:23,62 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:28:23,62 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:23,62 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:23,62 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:28:23,62 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:28:23,64 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,64 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,64 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,64 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,65 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates)] 05:28:23,65 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates)] 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates)] 05:28:23,65 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,65 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,66 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates)] 05:28:23,74 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,75 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,75 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,76 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,76 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,76 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,76 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,76 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,76 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,76 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 05:28:23,76 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 05:28:23,76 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=7c150ece-66a5-4eb1-9083-bd0cfa61db4a}, logIndex: 2, isSendImmediate: true 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=fbc00097-a9f1-4fe9-a1de-73a16fd3419b}, logIndex: 2, isSendImmediate: true 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=00ccc68f-5745-40d1-9255-9438e4a4bd78}, logIndex: 2, isSendImmediate: true 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,77 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,77 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,78 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 05:28:23,78 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 05:28:23,78 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=6f9f83fa-4306-47aa-ad51-20715982cd0d}, logIndex: 2, isSendImmediate: true 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,78 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] 05:28:23,79 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,79 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,79 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 3 05:28:23,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,85 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,86 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 3 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 3 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,86 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,86 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,86 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 05:28:23,86 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:28:23,86 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 05:28:23,87 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,87 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,87 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,87 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 3 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,88 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 05:28:23,90 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:28:23,91 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,91 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,92 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,92 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,92 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,92 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,93 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,93 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bee06c3 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bee06c3 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b2f6144 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b2f6144 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,93 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,93 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,93 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:28:23,93 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,93 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,93 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 05:28:23,94 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,94 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,94 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,94 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:28:23,94 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,94 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 05:28:23,95 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,96 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:23,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,96 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:28:23,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] 05:28:23,96 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] 05:28:23,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:23,97 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,97 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,97 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1693380386] 05:28:23,97 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,97 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1693380386] 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,98 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ff18caa 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ff18caa 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,98 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,98 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1396835413] 05:28:23,99 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:23,99 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1488854125] 05:28:23,99 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1488854125] 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,99 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1396835413] 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,99 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:28:23,99 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,99 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1378628571] 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28a8398e 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28a8398e 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,99 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1378628571] 05:28:23,99 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,100 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,100 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1766832072] 05:28:23,100 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,100 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,100 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:23,100 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#2125603897] 05:28:23,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#2125603897] 05:28:23,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1766832072] 05:28:23,100 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:28:23,100 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#148039628] 05:28:23,100 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#148039628] 05:28:23,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-403798589] 05:28:23,101 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:23,101 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2142905458] 05:28:23,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2142905458] 05:28:23,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-403798589] 05:28:23,101 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:28:23,101 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1843713479] 05:28:23,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1843713479] 05:28:23,102 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1821394753] 05:28:23,102 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:23,102 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] 05:28:23,102 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] 05:28:23,102 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1821394753] 05:28:23,102 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:28:23,102 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] 05:28:23,102 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] 05:28:23,103 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,103 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:23,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,103 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,108 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,108 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,108 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,109 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,109 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,109 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,109 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,110 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,110 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,110 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,112 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 05:28:23,113 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,113 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,113 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,114 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 05:28:23,115 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 2 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 1 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,116 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,116 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,116 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 2 05:28:23,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 1 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 1 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,117 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 05:28:23,117 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,117 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,117 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,117 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 2 05:28:23,118 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 1 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,118 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 05:28:23,118 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 05:28:23,120 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,120 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,120 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,122 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,123 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,124 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,129 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,137 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,137 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,137 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,137 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,137 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,137 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,138 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-2-shard-people-config_testFlipMemberVotingStates] 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-1-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,138 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:23,138 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:23,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,139 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,139 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,139 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,139 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 05:28:23,139 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,139 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,139 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,139 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,139 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,139 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,142 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 05:28:23,142 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,142 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,143 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,145 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,145 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,145 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,145 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,146 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,146 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,146 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,146 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,147 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 05:28:23,147 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 05:28:23,148 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,148 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,149 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,149 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,150 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,150 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,150 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 05:28:23,150 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,151 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 05:28:23,151 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 05:28:23,151 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,151 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 05:28:23,151 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,151 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,151 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,151 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,154 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,204 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,255 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,305 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,356 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,407 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,457 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,458 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 360.5 ms 05:28:23,458 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,458 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,459 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,459 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,459 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,459 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 366.0 ms 05:28:23,459 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,459 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 366.1 ms 05:28:23,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,459 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,464 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-3-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,464 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,464 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 364.6 ms 05:28:23,464 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,464 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,464 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,464 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,466 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,466 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,469 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 05:28:23,469 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:28:23,469 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 05:28:23,470 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,470 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 05:28:23,471 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,510 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] 05:28:23,511 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:23,511 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,512 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 05:28:23,512 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 05:28:23,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:28:23,512 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 05:28:23,512 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:28:23,512 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:28:23,513 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:23,513 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:28:23,513 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:28:23,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:23,514 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:28:23,514 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,515 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 05:28:23,515 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,515 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=59fe43eb-2def-42f4-8e32-39ee29784dfc}, logIndex: 3, isSendImmediate: true 05:28:23,516 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3e912871-2ad2-41a7-b2de-d70bbfda2020}, logIndex: 3, isSendImmediate: true 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,516 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 05:28:23,516 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] 05:28:23,516 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,517 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 05:28:23,517 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=16e1462a-dad9-4898-b95c-8adc1d4671d0}, logIndex: 3, isSendImmediate: true 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3b86e755-e173-4056-8624-6537f587695f}, logIndex: 3, isSendImmediate: true 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,517 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 05:28:23,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 05:28:23,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:28:23,518 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 05:28:23,518 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:28:23,519 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 05:28:23,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,519 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,522 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,522 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:23,523 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,523 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,523 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,523 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,524 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,524 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,524 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,524 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,525 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,526 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,526 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,526 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,526 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,527 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,528 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,528 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,528 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,528 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,528 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,528 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5290ebb 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5290ebb 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@605aecc3 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@605aecc3 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,529 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@330b36be 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@330b36be 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,529 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] 05:28:23,529 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,529 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,529 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,529 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,530 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,530 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,530 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-1393235739] 05:28:23,530 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] 05:28:23,530 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:23,530 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,530 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,530 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:28:23,530 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 05:28:23,530 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@397e5746 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@397e5746 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,531 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#535373033] 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,531 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,531 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,532 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,532 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,532 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,532 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,532 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#793291891] 05:28:23,532 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:28:23,532 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,532 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 05:28:23,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1693380386] 05:28:23,533 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-1693380386] 05:28:23,534 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1396835413] 05:28:23,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:23,534 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1488854125] 05:28:23,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,535 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,535 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,535 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,535 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 05:28:23,535 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,536 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1488854125] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 05:28:23,536 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,536 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,536 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,536 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,536 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1396835413] 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,536 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 05:28:23,536 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 05:28:23,536 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,536 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:28:23,536 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,537 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,537 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1378628571] 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,537 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,537 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,537 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1378628571] 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,537 PM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,538 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,538 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1766832072] 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 05:28:23,538 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,538 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#2125603897] 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:23,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#2125603897] 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,538 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,538 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1766832072] 05:28:23,538 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:28:23,538 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#148039628] 05:28:23,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#148039628] 05:28:23,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-403798589] 05:28:23,539 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:23,539 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2142905458] 05:28:23,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2142905458] 05:28:23,539 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-403798589] 05:28:23,539 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testFlipMemberVotingStates] 05:28:23,539 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,539 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 05:28:23,539 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,540 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:28:23,540 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,540 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,540 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,540 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,540 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,540 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:28:23,540 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1843713479] 05:28:23,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#1843713479] 05:28:23,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1821394753] 05:28:23,540 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:23,540 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] 05:28:23,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1526860725] 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,542 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,543 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,543 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,543 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,543 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-config_testFlipMemberVotingStates] 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,544 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,544 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,545 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-oper_testFlipMemberVotingStates] 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,546 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,546 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,546 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,546 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,546 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,547 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,548 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,549 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,549 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 05:28:23,549 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1821394753] 05:28:23,549 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,550 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:28:23,550 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 05:28:23,550 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 05:28:23,550 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,550 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 05:28:23,550 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 05:28:23,551 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,552 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,552 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,552 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#233041440] 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,552 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,553 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:23,554 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:28:23,554 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:28:23,554 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1924461547] 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:28:23,554 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,555 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,555 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,554 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 05:28:23,555 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,555 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,556 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,556 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,556 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,556 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,556 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,556 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,556 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,556 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,556 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,556 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,556 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,556 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,557 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,557 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,557 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,557 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,558 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,558 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,558 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,558 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply [term=4, success=false, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 05:28:23,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,558 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 05:28:23,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,559 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,559 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,559 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,560 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,560 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,560 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply [term=4, success=false, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 05:28:23,560 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,560 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 05:28:23,560 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,561 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader changed to member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,561 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-2-shard-people-config_testFlipMemberVotingStates in 32.79 ms 05:28:23,561 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,561 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,561 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,562 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-2-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,562 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,562 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader changed to member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,562 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,562 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-2-shard-cars-config_testFlipMemberVotingStates in 30.95 ms 05:28:23,562 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 05:28:23,562 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,563 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,563 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 05:28:23,563 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,563 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 05:28:23,563 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 05:28:23,563 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 05:28:23,563 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 05:28:23,563 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,564 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,564 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,564 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,564 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,564 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,564 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,566 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 05:28:23,567 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 05:28:23,569 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 05:28:23,569 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 05:28:23,607 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#301962674] 05:28:23,607 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:23,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,608 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,608 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:28:23,609 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onShutDown 05:28:23,609 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onShutDown 05:28:23,609 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStates 05:28:23,610 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,610 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStates 05:28:23,610 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,611 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:28:23,611 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:28:23,611 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:23,612 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,612 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,612 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:28:23,612 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onShutDown 05:28:23,612 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onShutDown 05:28:23,613 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStates 05:28:23,613 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStates 05:28:23,613 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,613 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,613 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:28:23,613 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:28:23,615 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:23,615 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:23,624 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:23,625 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:23,625 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:23,626 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:23,629 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:23,641 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:23,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,642 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:28:23,642 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onShutDown 05:28:23,642 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,642 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onShutDown 05:28:23,642 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,642 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51d8b93a 05:28:23,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51d8b93a 05:28:23,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@57dd3149 05:28:23,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@57dd3149 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,643 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,647 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-config_testFlipMemberVotingStates] 05:28:23,647 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,647 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:23,647 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,648 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 4 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,650 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,652 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,652 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 05:28:23,652 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,653 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 05:28:23,653 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,653 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 05:28:23,653 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,653 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,653 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,654 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,654 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,655 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Cluster state: [] 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,655 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,655 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Cluster state: [] 05:28:23,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 05:28:23,655 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,656 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 05:28:23,656 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,656 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 05:28:23,656 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:28:23,656 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,656 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,656 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 05:28:23,656 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,656 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,656 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 05:28:23,657 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,657 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,657 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,658 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,658 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,658 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,658 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,658 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,658 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 05:28:23,658 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,659 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,659 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,659 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 05:28:23,659 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 16.65 ms 05:28:23,659 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 05:28:23,659 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 05:28:23,660 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStates 05:28:23,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,660 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,660 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 19.56 ms 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 05:28:23,662 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,662 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 05:28:23,662 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStates 05:28:23,662 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,662 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:23,663 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:23,663 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:23,663 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,664 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,664 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onShutDown 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onShutDown 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17c997c3 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17c997c3 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,664 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55e3702e 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55e3702e 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:23,664 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,665 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,665 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,665 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,667 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,667 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,667 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-oper_testFlipMemberVotingStates] 05:28:23,667 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:23,668 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,668 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 05:28:23,668 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,668 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 05:28:23,668 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,668 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,669 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,669 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,669 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,669 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,669 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 05:28:23,669 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 05:28:23,672 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,672 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,672 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,673 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,673 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,673 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,674 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,674 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Cluster state: [] 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,674 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 05:28:23,674 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 05:28:23,674 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,675 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,675 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 05:28:23,675 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,675 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 05:28:23,675 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,676 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 05:28:23,676 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 05:28:23,677 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,677 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 05:28:23,677 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,677 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,677 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,677 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 05:28:23,677 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 05:28:23,678 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,678 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:23,678 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 14.14 ms 05:28:23,678 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 05:28:23,678 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,678 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 05:28:23,678 PM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 14.37 ms 05:28:23,678 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 05:28:23,678 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 05:28:23,678 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 05:28:23,679 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 05:28:23,679 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStates 05:28:23,679 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStates 05:28:23,679 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,679 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:23,679 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:23,679 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:23,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 05:28:23,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:23,682 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:23,682 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:23,861 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:28:24,629 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [3875736336021850963] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:24,701 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:24,701 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6341693480696104090] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:24,702 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:24,704 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:24.709] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:24,718 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 05:28:24,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 05:28:24,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 05:28:24,719 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:28:24,719 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onShutDown 05:28:24,719 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onShutDown 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71364cec 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71364cec 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e6298a5 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4e6298a5 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:24,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:24,720 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:24,721 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:24,721 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:25,809 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 05:28:25,809 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 05:28:25,809 PM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.090 s 05:28:25,809 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.090 s 05:28:25,809 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 05:28:25,809 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 05:28:25,809 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 05:28:25,809 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 05:28:25,809 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 05:28:25,809 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 05:28:25,810 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStates 05:28:25,810 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStates 05:28:25,810 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:25,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:25,811 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:25,812 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:25,812 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 05:28:25,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:25,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onShutDown 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:25,813 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:25,813 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onShutDown 05:28:25,813 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@652cfd2c 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@652cfd2c 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:25,813 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:25,813 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:25,813 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 05:28:25,813 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16cfe622 05:28:25,814 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@16cfe622 05:28:25,814 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 05:28:25,814 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 05:28:25,814 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:25,814 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:28:26,909 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 05:28:26,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 05:28:26,909 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.096 s 05:28:26,909 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 05:28:26,909 PM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.096 s 05:28:26,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 05:28:26,910 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 05:28:26,910 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 05:28:26,910 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 05:28:26,910 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 05:28:26,911 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStates 05:28:26,911 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStates 05:28:26,911 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:26,911 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Aborting 0 pending queued transactions 05:28:26,912 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:26,912 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:26,915 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:26,915 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:26,921 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:26.925] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:26,959 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:26,976 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5076619992974005882] 05:28:26,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:26,982 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:26,982 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:26,985 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:26,985 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:26,992 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:26,997 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:26,997 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:27,2 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,5 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 05:28:27,6 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:27,13 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,16 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:27,17 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:27,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,18 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,18 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,18 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,18 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:27,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@70c22d83 05:28:27,24 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 05:28:27,25 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,25 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,26 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1406967589] 05:28:27,27 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testModuleShardLeaderMovement, persistent : true 05:28:27,27 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,27 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 05:28:27,28 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testModuleShardLeaderMovement, persistent : true 05:28:27,28 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testModuleShardLeaderMovement with journal batch size 1 05:28:27,28 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1225653065 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1421306939 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-997666177] 05:28:27,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1225653065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,30 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1406967589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,30 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1421306939] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,30 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-997666177] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,34 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#330955034] 05:28:27,35 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#330955034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:27,36 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-config_testModuleShardLeaderMovement 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,36 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 05:28:27,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:27,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 05:28:27,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:27,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 05:28:27,37 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 05:28:27,37 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:27,37 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:27,38 PM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:27,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:27,38 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 05:28:27,38 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:27,38 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 05:28:27,39 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:27,39 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 05:28:27,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 05:28:27,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 05:28:27,39 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:27,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:27,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:27,39 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:27,39 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 05:28:27,39 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:27,39 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 05:28:27,39 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:27,75 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] 05:28:27,75 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:27,75 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:27,75 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,77 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,80 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 05:28:27,82 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-997680411] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,83 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,85 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,86 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:27,86 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@245cbfc5 05:28:27,86 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 05:28:27,87 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,87 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,88 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 05:28:27,88 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,88 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-863513904 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,88 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 05:28:27,89 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#1043734261] 05:28:27,89 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-863513904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,89 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,89 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,89 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122] 05:28:27,89 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,89 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:27,89 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:27,89 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,89 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:27,90 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 05:28:27,90 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:27,90 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,90 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,90 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#1745701209] 05:28:27,90 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:27,90 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 05:28:27,90 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testModuleShardLeaderMovement, persistent : true 05:28:27,91 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#1755889929 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,91 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-1399830747] 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testModuleShardLeaderMovement with journal batch size 1 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:27,91 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:27,92 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:27,92 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,92 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 05:28:27,92 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 05:28:27,92 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 05:28:27,92 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:27,92 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:27,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 05:28:27,92 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:27,93 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:27,93 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 05:28:27,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:27,137 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-997680411] 05:28:27,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:27,138 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122] 05:28:27,151 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:27,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6830329070007890078] 05:28:27,168 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:27,173 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:27,173 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:27,173 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:27,173 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:27,185 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,186 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,189 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 05:28:27,193 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:27,195 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,204 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:27,207 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:27,208 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3817db8c 05:28:27,208 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 05:28:27,210 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,210 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,212 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,215 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 05:28:27,218 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:27,218 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1757247703] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,219 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:27,222 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:27,223 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32efb0fc 05:28:27,223 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 05:28:27,223 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,238 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:27,255 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8461543773041471977] 05:28:27,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:28:27,259 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:27,259 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:28:27,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:27,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:27,280 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,282 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,284 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 05:28:27,301 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:27,302 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 05:28:27,302 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,304 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:28:27,306 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:28:27,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3ed875b0 05:28:27,306 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,307 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:27,308 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:27,310 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 05:28:27,313 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1907313122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:27,314 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:28:27,315 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:28:27,316 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:27,317 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34201ec0 05:28:27,317 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 05:28:27,318 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:27,320 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:27,320 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,320 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:27,320 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,320 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:27,344 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:28:27,345 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:27,346 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,346 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:27,346 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:27,346 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,2 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:28:28,2 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:28,4 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,21 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,21 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,21 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,21 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,21 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,21 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,21 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,21 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:28,210 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,224 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:28,224 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:28,225 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,225 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 05:28:28,228 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,229 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064, primaryVersion=13] 05:28:28,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,232 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testModuleShardLeaderMovement, persistent : true 05:28:28,234 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 05:28:28,234 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,234 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#47340949] 05:28:28,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#922763964 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,234 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#47340949] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,234 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#922763964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,235 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:28,236 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:28,236 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:28,236 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,236 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 05:28:28,236 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,236 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 05:28:28,236 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 05:28:28,236 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3bd8dbaf, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=null] 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:28,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testModuleShardLeaderMovement: 229 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 05:28:28,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,241 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:28,241 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,241 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:28,241 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:28,241 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,242 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 05:28:28,242 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,242 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,242 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@9f7ed61, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:28,242 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:28,243 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,243 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 05:28:28,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1789003401] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 05:28:28,244 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1789003401] 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 438.6 μs 05:28:28,244 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,244 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#1789003401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,244 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 106.0 μs 05:28:28,244 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,245 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,245 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=c3f8e810-a829-42c0-96e4-69ff35cdc392}, logIndex: 0, isSendImmediate: true 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,246 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,247 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 05:28:28,247 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,248 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING]] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,249 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,249 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,250 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 0 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,250 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:28,251 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:28,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 05:28:28,251 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064 05:28:28,251 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 05:28:28,252 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,252 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 05:28:28,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:28,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,252 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:28,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 05:28:28,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,252 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 0 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 0 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 05:28:28,253 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,253 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] 05:28:28,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:28,254 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:28,254 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,254 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1757247703] 05:28:28,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:28,254 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 05:28:28,255 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:28,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:28,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 05:28:28,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122, primaryVersion=13] 05:28:28,261 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,261 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 05:28:28,261 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#187065830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,262 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-844801058] 05:28:28,262 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1092370439 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,262 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1092370439] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,262 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-844801058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,262 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 05:28:28,263 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 05:28:28,264 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 05:28:28,264 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:28,264 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:28,264 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,264 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 05:28:28,264 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:28,264 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,264 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 05:28:28,265 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5b598321, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=null] 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: 229 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 05:28:28,265 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,267 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:28,267 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,267 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:28,267 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:28,267 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,268 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 05:28:28,268 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1657d991, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 05:28:28,268 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#505893200] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 05:28:28,269 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#505893200] 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 372.1 μs 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,269 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,269 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 108.7 μs 05:28:28,269 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 05:28:28,269 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#505893200] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,270 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8f8df58c-aab5-4bd9-8816-28365cfccbb2}, logIndex: 0, isSendImmediate: true 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,271 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 05:28:28,272 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:28,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122 05:28:28,273 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 05:28:28,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING]] 05:28:28,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 05:28:28,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,274 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,274 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 05:28:28,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:28,274 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1757247703] 05:28:28,274 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,274 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,274 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:28,274 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#187065830] 05:28:28,274 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#187065830] 05:28:28,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,275 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 05:28:28,275 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:28,275 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 0 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:28,275 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:28,276 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 05:28:28,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,276 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 05:28:28,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064, primaryVersion=13] 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 0 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 0 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 05:28:28,278 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 05:28:28,279 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:28:28,281 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,281 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testModuleShardLeaderMovement, persistent : true 05:28:28,283 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 05:28:28,283 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,283 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#1048280094] 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:28,283 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,284 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#859868482 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,284 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#1048280094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5df9d95d, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testModuleShardLeaderMovement: 229 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,284 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:28,284 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:28,284 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 05:28:28,285 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,285 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 05:28:28,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:28:28,285 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 05:28:28,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 05:28:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 05:28:28,287 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 05:28:28,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,287 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:28,287 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,288 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,288 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 05:28:28,288 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,288 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@39e52759, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1201904065] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 05:28:28,289 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1201904065] 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 433.0 μs 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 05:28:28,289 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,290 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 05:28:28,290 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,290 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 80.39 μs 05:28:28,290 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,290 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 05:28:28,291 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=3a68a9ab-037e-4351-b7a8-0caf10a400bd}, logIndex: 1, isSendImmediate: true 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 05:28:28,293 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 2 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 05:28:28,295 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,296 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,296 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,296 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,296 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,296 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1907313122] 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:28,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,297 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:28:28,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 05:28:28,297 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,297 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,297 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:28,297 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 05:28:28,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#859868482] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1201904065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,297 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,297 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 05:28:28,297 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 1 05:28:28,297 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,297 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,298 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,299 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 05:28:28,300 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,300 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,306 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:28,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:28,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 05:28:28,310 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:28,310 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122, primaryVersion=13] 05:28:28,314 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 05:28:28,314 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1922058844] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,315 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#216888401] 05:28:28,315 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1526488983 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,315 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1526488983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,315 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#216888401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,315 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 05:28:28,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:28,316 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:28,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:28,316 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 05:28:28,316 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:28:28,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 05:28:28,318 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 05:28:28,318 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 05:28:28,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 05:28:28,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 05:28:28,318 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,318 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 05:28:28,319 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:28,319 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,319 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 05:28:28,319 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@65524d55, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: 229 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-3-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 05:28:28,320 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 05:28:28,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 05:28:28,321 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,322 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 05:28:28,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:28,322 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,323 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 05:28:28,323 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@511a215, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:28,324 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 05:28:28,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:28,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-687133237] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 05:28:28,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-687133237] 05:28:28,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 655.1 μs 05:28:28,325 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 05:28:28,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 05:28:28,325 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-687133237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:28,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 150.7 μs 05:28:28,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 05:28:28,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 05:28:28,326 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f71c8aa4-90ae-4a8c-aa33-2014ca2f29b0}, logIndex: 1, isSendImmediate: true 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,327 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 05:28:28,328 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 2 05:28:28,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 05:28:28,330 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 05:28:28,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 1 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 05:28:28,331 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,331 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-oper_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,331 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,332 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,332 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 05:28:28,332 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-1450881122 05:28:28,332 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 05:28:28,334 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,334 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 05:28:28,333 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 05:28:28,333 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 05:28:28,334 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:28,334 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,335 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 05:28:28,335 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 05:28:28,335 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:28,336 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1907313122] 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 05:28:28,336 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:28:28,336 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1922058844] 05:28:28,336 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#1922058844] 05:28:28,337 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] 05:28:28,337 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:28,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,337 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,338 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] 05:28:28,338 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:28,339 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,339 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,339 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,339 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,339 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,340 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,342 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard cars, datastoreType Config 05:28:28,342 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] 05:28:28,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:28,342 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,344 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,345 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 05:28:28,345 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 05:28:28,346 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] 05:28:28,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:28,347 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,347 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,351 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] 05:28:28,352 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:28,352 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,352 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,352 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,352 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard cars, datastoreType Config 05:28:28,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] 05:28:28,353 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:28,353 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,353 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,353 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 05:28:28,353 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,360 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-2-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/member-2-shard-cars-config_testModuleShardLeaderMovement$e]] 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7325743c 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7325743c 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,362 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,364 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,364 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,364 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,364 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,364 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 05:28:28,366 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,368 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,369 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:28:28,369 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,369 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,369 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:28,369 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,369 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,370 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 1 - switching to Follower 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,370 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,371 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 05:28:28,371 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,371 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 05:28:28,371 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:28:28,371 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:28:28,371 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,371 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:28,372 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 05:28:28,372 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,373 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 11.80 ms 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,373 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,373 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,373 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,373 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,377 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,378 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,378 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 05:28:28,378 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1359824588] 05:28:28,378 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:28,379 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,379 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#314508497] 05:28:28,379 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#99492218] 05:28:28,379 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:28,379 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,379 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#2019108064] 05:28:28,380 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,380 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,380 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,380 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,380 PM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard cars, datastoreType Config 05:28:28,380 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,380 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,380 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,380 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,380 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 05:28:28,380 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-3-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[akka://cluster-test@127.0.0.1:2557/temp/member-3-shard-cars-config_testModuleShardLeaderMovement$e]] 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 05:28:28,382 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@209566ab 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@209566ab 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,382 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,383 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,383 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,384 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,384 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 05:28:28,385 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 05:28:28,385 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,385 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,385 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 05:28:28,385 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 05:28:28,386 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:28,386 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,386 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,387 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:28,387 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,387 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 2 - updating term 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,388 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:28,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 2 - switching to Follower 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,389 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:28,389 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:28,390 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:28,390 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,390 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 05:28:28,390 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,390 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,391 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 9.098 ms 05:28:28,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,391 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,391 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,390 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-3-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,390 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,391 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,391 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,391 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,392 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,392 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,392 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 05:28:28,392 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,392 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1309515096] 05:28:28,392 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:28,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,393 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#401322638] 05:28:28,393 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,393 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:28,393 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onShutDown 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 05:28:28,394 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69a20bdf 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@69a20bdf 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 05:28:28,394 PM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 05:28:28,395 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,395 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,395 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,396 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,396 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,396 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 3 - updating term 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 05:28:28,397 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,398 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,398 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 3 - switching to Follower 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:28:28,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 05:28:28,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 05:28:28,398 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 05:28:28,398 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 05:28:28,399 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 05:28:28,399 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 05:28:28,399 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 05:28:28,399 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 05:28:28,399 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,400 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:28,400 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 6.000 ms 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 05:28:28,400 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,400 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 05:28:28,400 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,400 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 05:28:28,400 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testModuleShardLeaderMovement 05:28:28,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:28,401 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:28:28,401 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:28:28,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 05:28:28,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:28,402 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:28,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,402 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 05:28:28,402 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 05:28:28,403 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,403 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:28,403 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:28:28,403 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:28:28,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 05:28:28,404 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:28,404 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:28,409 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:28,409 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:28,410 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:28,410 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:28,413 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:28,429 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:28,430 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,430 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 05:28:28,430 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onShutDown 05:28:28,431 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testModuleShardLeaderMovement 05:28:28,431 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:28,431 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:28,431 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:28,432 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:28,432 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,432 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 05:28:28,432 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 05:28:28,432 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testModuleShardLeaderMovement 05:28:28,432 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:28,433 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:28,433 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:28,434 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:28,434 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:29,21 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:28:29,421 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [8461543773041471977] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:29,449 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:29,451 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6830329070007890078] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:29,451 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:29,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:29.456] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:29,466 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 05:28:29,467 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:29,467 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 05:28:29,467 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:28:29,467 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onShutDown 05:28:29,467 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 05:28:29,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onShutDown 05:28:29,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7c7aa652 05:28:29,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7c7aa652 05:28:29,467 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63316ce7 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63316ce7 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-people-config_testModuleShardLeaderMovement 05:28:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:34,122 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 05:28:34,122 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 05:28:34,123 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:34,123 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:34,123 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:34,123 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:34,124 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:36,80 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:37,39 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:28:38,630 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 05:28:38,630 PM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.163 s 05:28:38,630 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 05:28:38,630 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 05:28:38,631 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer expired 05:28:38,631 PM [cluster-test-shard-dispatcher-21] [WARN] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 05:28:38,631 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 05:28:38,631 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:38,631 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:38,632 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 05:28:38,632 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:38,632 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 05:28:38,632 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 05:28:38,632 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testModuleShardLeaderMovement 05:28:38,633 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:38,633 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:38,636 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:38,636 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 05:28:38,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating leader transfer 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f9995c 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f9995c 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:38,637 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onShutDown 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6272bffc 05:28:38,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6272bffc 05:28:38,638 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testModuleShardLeaderMovement 05:28:38,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:38,638 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:38,638 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement] 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 05:28:42,341 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.704 s 05:28:42,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 05:28:42,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 05:28:42,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 05:28:42,342 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 05:28:42,342 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 05:28:42,342 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 05:28:42,343 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Aborting 0 pending queued transactions 05:28:42,343 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:42,343 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:42,345 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:42,345 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:42,350 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:42.353] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:42,387 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:42,402 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7621428704504648753] 05:28:42,403 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:42,406 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:42,406 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:42,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:42,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:42,426 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,428 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,431 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 05:28:42,435 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:42,435 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:42,435 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:42,443 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-982671531] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,443 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:42,449 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:42,449 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 05:28:42,449 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 05:28:42,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,450 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:42,478 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f070045 05:28:42,479 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 05:28:42,479 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 05:28:42,481 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,481 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:42,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 05:28:42,482 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,482 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,482 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#983421407] 05:28:42,483 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardReplica, persistent : true 05:28:42,483 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,483 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-1371858348] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,484 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardReplica, persistent : true 05:28:42,483 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#2011709478] 05:28:42,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#983421407] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,485 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 05:28:42,485 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#696288788 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 05:28:42,485 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1603304414] 05:28:42,483 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-1371858348 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 05:28:42,486 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardReplica with journal batch size 1 05:28:42,489 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:42,490 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#2011709478] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,490 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#696288788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,490 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardReplica with journal batch size 1 05:28:42,490 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#1603304414] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,490 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1862872701] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,492 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,492 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,492 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:42,492 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,492 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,492 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,492 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,492 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 05:28:42,492 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 05:28:42,492 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,492 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,492 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:42,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 05:28:42,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:42,492 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 05:28:42,492 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 05:28:42,493 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,493 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,493 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,493 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,493 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 05:28:42,493 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,493 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 05:28:42,504 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:42,505 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1f4ffe5 05:28:42,505 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 05:28:42,506 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,506 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,507 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardReplica, persistent : true 05:28:42,508 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardReplica with journal batch size 1 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-1234584938] 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,508 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,509 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1090685773] 05:28:42,509 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardReplica, persistent : true 05:28:42,509 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardReplica with journal batch size 1 05:28:42,509 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-821771104 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,510 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/shard-member-1:snapshot-read#1564937339] 05:28:42,510 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-1234584938] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,510 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/shard-member-1:snapshot-read#603627350] 05:28:42,510 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1090685773] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,512 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#1772555206 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:42,513 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 05:28:42,513 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,513 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,513 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 05:28:42,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,513 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,513 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:42,513 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 05:28:42,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,514 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 05:28:42,514 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,514 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,514 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 05:28:42,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:42,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2288199046880271409] 05:28:42,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:42,532 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:42,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:42,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:42,535 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:42,542 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,552 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,554 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 05:28:42,560 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1104690727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,566 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:42,572 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:42,572 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 05:28:42,572 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 05:28:42,593 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:42,595 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:42,596 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:42,596 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,596 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,599 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4d3c885a 05:28:42,599 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 05:28:42,600 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,600 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,600 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 05:28:42,602 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,604 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 05:28:42,606 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:42,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-2-shard-cars-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] 05:28:42,606 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 05:28:42,607 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:42,607 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-2-shard-people-config_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#1218704488] 05:28:42,607 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 05:28:42,607 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,608 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-869640413 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 05:28:42,608 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#2110610353] 05:28:42,608 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-869640413] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,608 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#2110610353] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,608 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 05:28:42,608 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#485235440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,609 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:42,609 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 05:28:42,609 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,609 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,609 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#1218704488] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,610 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-457016785 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 05:28:42,610 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:42,610 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 05:28:42,616 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:42,616 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 05:28:42,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-457016785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,616 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 05:28:42,616 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,616 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 05:28:42,617 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,617 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,617 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,617 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 05:28:42,617 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,617 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,617 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,619 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-668231887] 05:28:42,619 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 05:28:42,619 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-668231887] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,621 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:28:42,621 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,621 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,621 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,621 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 05:28:42,631 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:42,632 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52b7dc0c 05:28:42,632 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 05:28:42,636 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,636 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:42,637 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 05:28:42,637 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 05:28:42,637 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,637 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:42,637 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-2-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1753877236] 05:28:42,637 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 05:28:42,638 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 05:28:42,638 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 05:28:42,638 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,638 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:42,638 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-2-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1654560591] 05:28:42,638 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-1308892015] 05:28:42,639 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1753877236] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,639 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1654560591] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,639 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#36830649 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 05:28:42,639 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-1774738304] 05:28:42,639 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-480433843 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 05:28:42,640 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,640 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,641 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,641 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,641 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 05:28:42,641 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:28:42,641 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 05:28:42,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,641 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,641 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,641 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 05:28:42,641 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,641 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:42,641 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,641 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 05:28:42,641 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 05:28:42,641 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,641 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,641 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,641 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 05:28:42,645 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:42,658 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7576886986274860974] 05:28:42,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:28:42,664 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:42,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:28:42,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:42,666 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:42,670 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,672 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,675 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 05:28:42,681 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1647811102] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,682 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:28:42,688 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:28:42,688 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 05:28:42,688 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 05:28:42,701 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:42,703 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@68534468 05:28:42,703 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 05:28:42,704 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,705 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,706 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 05:28:42,706 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:42,707 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:28:42,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:42,711 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 05:28:42,713 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-275543333] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,714 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:28:42,715 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardReplica, persistent : true 05:28:42,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-216592966] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,715 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#1221061681] 05:28:42,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-298215877 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 05:28:42,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-298215877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,716 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardReplica with journal batch size 1 05:28:42,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 05:28:42,716 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:42,717 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardReplica, persistent : true 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:28:42,717 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#730758501 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 05:28:42,717 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 05:28:42,717 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#721309996] 05:28:42,718 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardReplica with journal batch size 1 05:28:42,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 05:28:42,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:42,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,720 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#1221061681] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,720 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-1187596554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,720 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#730758501] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,720 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#721309996] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,724 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,724 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 05:28:42,724 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,724 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardReplica is no longer the leader 05:28:42,724 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:28:42,724 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,724 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardReplica is no longer the leader 05:28:42,725 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 05:28:42,725 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:42,725 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:42,725 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:42,725 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 05:28:42,725 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 05:28:42,749 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:42,750 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@496bda44 05:28:42,751 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 05:28:42,752 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:42,752 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 05:28:42,752 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:42,754 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardReplica, persistent : true 05:28:42,754 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-1049746111] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,754 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:42,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-1049746111] 05:28:42,754 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 05:28:42,755 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardReplica with journal batch size 1 05:28:42,755 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardReplica, persistent : true 05:28:42,755 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/shard-member-3:snapshot-read#862244303] 05:28:42,755 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#-573066097 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 05:28:42,755 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:42,755 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#756927214] 05:28:42,756 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-990256470] 05:28:42,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#528640502 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 05:28:42,756 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardReplica with journal batch size 1 05:28:42,757 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#756927214] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,757 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:28:42,757 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardReplica is no longer the leader 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:42,757 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,757 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,757 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 05:28:42,758 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:42,758 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:42,758 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:28:42,758 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 05:28:42,758 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,758 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardReplica is no longer the leader 05:28:42,759 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 05:28:42,759 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,759 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:42,759 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 05:28:42,838 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:42,838 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 05:28:42,838 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 05:28:42,838 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:42,838 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 05:28:42,839 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:42,839 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,839 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 05:28:42,839 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 05:28:42,839 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,848 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 05:28:42,849 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 05:28:42,849 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 05:28:42,877 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 05:28:42,878 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 05:28:42,878 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:42,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 05:28:42,898 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 05:28:42,899 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 05:28:43,217 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:43,218 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 05:28:43,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,238 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,238 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 05:28:43,238 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:43,238 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 05:28:43,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:43,248 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 05:28:43,419 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:28:43,419 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,420 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:43,420 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,421 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:43,422 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,431 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,431 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:43,574 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:43,575 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:43,588 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,588 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:43,588 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 05:28:43,588 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardReplica 05:28:43,588 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardReplica 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:43,590 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,590 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,590 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,590 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,590 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,591 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,591 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,591 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:43,591 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,592 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:43,592 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:43,592 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 05:28:43,592 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,592 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,593 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:43,593 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:43,593 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 05:28:43,593 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 05:28:43,593 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,593 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 05:28:43,594 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:43,594 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 05:28:43,594 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:43,594 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 05:28:43,594 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,595 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,595 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,595 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,595 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:43,608 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 05:28:43,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardReplica 05:28:43,608 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardReplica 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:43,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:43,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 05:28:43,613 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,613 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 05:28:43,614 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 05:28:43,614 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:43,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:43,615 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 05:28:43,615 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 05:28:43,615 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,615 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,616 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,616 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,616 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,616 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,637 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 05:28:43,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardReplica 05:28:43,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,639 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,639 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,640 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,640 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,640 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,640 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,640 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,640 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,641 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:43,642 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:43,642 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 05:28:43,642 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 05:28:43,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 05:28:43,642 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:43,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,643 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,643 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 05:28:43,643 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 05:28:43,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,643 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:43,643 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:43,644 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,644 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,644 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,644 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,647 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 05:28:43,647 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:43,647 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 05:28:43,647 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardReplica 05:28:43,647 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardReplica 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:43,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:43,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 05:28:43,650 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 05:28:43,650 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,651 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 05:28:43,651 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:43,651 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 05:28:43,651 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:43,652 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:43,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:43,652 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 05:28:43,652 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,652 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:43,652 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:43,652 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:43,652 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 05:28:43,652 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 05:28:43,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-982671531] 05:28:43,652 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:43,652 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:43,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:43,653 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,653 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,653 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1104690727] 05:28:43,653 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:43,654 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] 05:28:43,654 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 05:28:43,654 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] 05:28:43,654 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:43,654 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1647811102] 05:28:43,654 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:43,654 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-216592966] 05:28:43,654 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-216592966] 05:28:43,657 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-3, datastoreType Config 05:28:43,658 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 05:28:43,660 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:43,660 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 05:28:43,661 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 05:28:43,663 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'}, state: Idle 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=68da6b48-df82-4371-9019-fa7e782f0c83}, logIndex: 0, isSendImmediate: true 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardReplica 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 05:28:43,666 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'} 05:28:43,667 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:43,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 05:28:43,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Number of entries to be appended = 1 05:28:43,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:43,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 05:28:43,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:43,669 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, votingState=VOTING]] 05:28:43,669 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Log size is now 1 05:28:43,669 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:43,669 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 05:28:43,669 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 05:28:43,670 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: 0, nextIndex: 1 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Setting last applied to 0 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:43,670 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 05:28:43,672 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:43,672 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Commit index set to 0 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Setting last applied to 0 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 05:28:43,673 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 05:28:43,674 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-982671531] 05:28:43,674 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:43,674 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:43,674 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:43,675 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-216592966] with 27000 ms timeout 05:28:43,675 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1104690727] 05:28:43,675 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:43,675 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] 05:28:43,675 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onShutDown 05:28:43,675 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-904518952] 05:28:43,675 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#1647811102] 05:28:43,675 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardReplica 05:28:43,675 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:43,676 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 05:28:43,676 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 05:28:43,676 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-216592966] 05:28:43,677 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:43,677 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 05:28:43,677 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:43,677 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:28:43,677 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:28:43,678 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2559] as [Down] 05:28:43,678 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:43,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 05:28:43,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 05:28:43,678 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:28:43,678 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 05:28:43,679 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 05:28:43,679 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 05:28:43,679 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 05:28:43,679 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:43,679 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:43,679 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:43,680 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:43,680 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 05:28:43,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:43,681 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:43,682 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:43,682 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:43,683 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 05:28:43,683 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:43,683 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:43,686 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 05:28:43,689 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:43,689 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:43,689 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:43,689 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Shutting down... 05:28:43,690 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:43,691 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Successfully shut down 05:28:43,692 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:43,712 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:43,724 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7418068649050882059] 05:28:43,724 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:43,732 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:43,732 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:43,735 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:43,735 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:43,746 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:43,747 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:43,750 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 05:28:43,759 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:43,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2000655999] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,764 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:43,765 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 05:28:43,766 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 05:28:43,771 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 05:28:43,772 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@704ebca3 05:28:43,772 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:43,772 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 05:28:43,772 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:43,773 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 05:28:43,773 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 05:28:43,774 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1662532197 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 05:28:43,774 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,774 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-1662532197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,774 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 05:28:43,774 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-1996666258] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,775 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1012662788] 05:28:43,775 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#74635129 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 05:28:43,783 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 05:28:43,783 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:43,783 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1012662788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,783 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#74635129] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,783 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1104515667] 05:28:43,783 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#1104515667] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,785 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:43,788 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:43,789 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:43,789 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 05:28:43,789 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 05:28:43,789 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:28:43,789 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 05:28:43,790 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received ReplicatedLogEntry for recovery: index: 0, size: 221 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=null, votingState=VOTING]] 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received apply journal entries for recovery, applying to state: 0 to 0 05:28:43,790 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 05:28:43,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:43,790 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 05:28:43,791 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in 129.7 μs - Switching actor to Follower - last log index = 0, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 1 05:28:43,791 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:43,791 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 05:28:43,791 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:43,791 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 05:28:43,791 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 05:28:43,792 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardReplica 05:28:43,792 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:43,793 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1939365068] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,794 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 05:28:43,799 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:43,799 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,799 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 05:28:43,811 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6190d460 05:28:43,811 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 05:28:43,812 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:43,812 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,812 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 05:28:43,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 05:28:43,813 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 05:28:43,814 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#738742673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,815 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#1644201531] 05:28:43,815 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-826090220 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 05:28:43,815 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 05:28:43,815 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-826090220] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:43,815 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:43,816 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:43,817 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-784104240 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 05:28:43,817 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:43,817 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 05:28:43,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:28:43,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 05:28:43,817 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 05:28:43,817 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:43,817 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-365211150] 05:28:43,817 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 05:28:43,818 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 05:28:43,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 05:28:43,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:28:43,818 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 05:28:43,819 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:43,819 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:43,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:43,819 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:43,819 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 05:28:43,819 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 05:28:43,819 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 05:28:43,819 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica 05:28:43,819 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 05:28:43,911 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 05:28:43,913 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:43,929 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 05:28:43,931 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 05:28:43,960 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 05:28:43,961 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:44,233 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:44,251 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:44,280 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:44,439 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:28:44,551 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:44,571 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:44,601 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:44,698 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2288199046880271409] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:44,699 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2288199046880271409] but current is [-7418068649050882059]. Reason: ActorSystem terminated 05:28:44,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardReplica peerAddress = null 05:28:44,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:44,758 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 05:28:44,758 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 05:28:44,758 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardReplica peerAddress = null 05:28:44,759 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:44,759 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 05:28:44,759 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardReplica peerAddress = null 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardReplica peerAddress = null 05:28:44,759 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 05:28:44,759 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 05:28:44,760 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 05:28:44,764 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:44,764 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 05:28:44,764 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 05:28:44,764 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 05:28:44,765 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#1666787444]] intended for system uid [-2288199046880271409], self uid is [-7418068649050882059] 05:28:44,769 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:44,769 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:44,769 PM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1153, lastApplied : -1, commitIndex : -1 05:28:44,769 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1173, lastApplied : -1, commitIndex : -1 05:28:44,769 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1125, lastApplied : -1, commitIndex : -1 05:28:44,770 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1096, lastApplied : 0, commitIndex : 0 05:28:44,772 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:44,772 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:44,871 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:44,890 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:44,920 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:45,190 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:45,190 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:45,192 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 421, lastApplied : -1, commitIndex : -1 05:28:45,194 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 05:28:45,210 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:45,210 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:45,211 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 440, lastApplied : -1, commitIndex : -1 05:28:45,211 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:45,239 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:45,239 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:45,241 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 470, lastApplied : -1, commitIndex : -1 05:28:45,242 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:45,249 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 475, lastApplied : 0, commitIndex : 0 05:28:45,511 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 05:28:45,511 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:45,531 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:45,532 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:45,559 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:45,560 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:45,571 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 05:28:45,769 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:45,830 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:45,831 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:45,851 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:45,852 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:45,881 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:45,881 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:45,889 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 05:28:46,152 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:46,152 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:46,171 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,171 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,200 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:46,200 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,209 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 05:28:46,472 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,473 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:46,489 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:46,490 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,519 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,519 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,529 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:46,790 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:46,791 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:46,795 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 05:28:46,810 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:46,810 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:46,842 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 05:28:46,843 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 05:28:46,851 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 05:28:47,110 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 05:28:47,112 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:47,131 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:47,131 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:47,159 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:47,160 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 05:28:47,172 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 05:28:47,432 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:47,432 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 05:28:47,450 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:47,450 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:47,480 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:47,481 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:47,490 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 05:28:47,751 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:47,752 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:47,770 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:47,770 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:47,802 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:47,803 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:47,808 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:47,810 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:48,71 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:48,72 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:48,90 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:48,90 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:48,121 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:48,122 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:48,129 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 05:28:48,390 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:48,390 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:48,410 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:48,410 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:48,440 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:48,440 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:48,450 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 05:28:48,489 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1244088413]] intended for system uid [-2288199046880271409], self uid is [-7418068649050882059] 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:28:48,708 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,708 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 05:28:48,708 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,711 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:28:48,712 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,712 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 05:28:48,713 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 05:28:48,713 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:48,713 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,728 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#1666787444]] intended for system uid [-2288199046880271409], self uid is [-7418068649050882059] 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:28:48,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,728 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 05:28:48,729 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,731 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:48,731 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 05:28:48,731 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 05:28:48,732 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,732 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,732 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:28:48,732 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,732 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 05:28:48,732 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 05:28:48,732 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:48,734 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 324, lastApplied : -1, commitIndex : -1 05:28:48,758 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,758 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 05:28:48,758 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,758 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,759 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:28:48,759 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,759 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 05:28:48,759 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,762 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:48,762 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 05:28:48,762 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 05:28:48,763 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,763 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:48,763 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:28:48,764 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,764 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 05:28:48,764 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 05:28:48,764 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:48,764 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,768 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 05:28:48,768 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 05:28:48,769 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:48,770 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 05:28:48,770 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 05:28:48,770 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:48,770 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:28:48,771 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:48,771 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 05:28:48,771 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 05:28:48,771 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:48,827 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:49,32 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:49,33 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,50 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 05:28:49,50 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 05:28:49,81 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 05:28:49,81 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 05:28:49,90 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 05:28:49,351 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:49,351 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:49,369 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,370 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,400 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,400 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,410 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 05:28:49,539 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Down)]. 05:28:49,540 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,540 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,556 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,556 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,670 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,671 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,690 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,690 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:49,720 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,720 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:49,730 PM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:49,799 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2559] 05:28:49,800 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,801 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,802 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2288199046880271409] but current is [-7418068649050882059]. Reason: Cluster member removed, previous status [Down] 05:28:49,848 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:28:49,907 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,908 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:49,908 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2288199046880271409] but current is [-7418068649050882059]. Reason: Cluster member removed, previous status [Down] 05:28:49,992 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:49,993 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:50,9 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,9 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,40 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,40 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,49 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 05:28:50,312 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,313 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:50,330 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:50,331 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:28:50,359 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,360 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,369 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:50,630 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:28:50,631 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:28:50,650 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,651 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:28:50,679 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,679 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:28:50,689 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 05:28:50,869 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:50,873 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:50,873 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:50,873 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:50,873 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 05:28:50,874 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 05:28:50,874 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 05:28:50,875 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 05:28:50,875 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 05:28:50,875 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 05:28:50,875 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2000655999] 05:28:50,875 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:50,875 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] 05:28:50,877 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] 05:28:50,877 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-2, datastoreType Config 05:28:50,877 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 05:28:50,878 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:50,878 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4056f5a6] 05:28:50,881 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136 for shard member-2-shard-cars-config_testRemoveShardReplica 05:28:50,882 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'}, state: Idle 05:28:50,882 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 05:28:50,882 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:50,882 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:50,882 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'} 05:28:50,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 05:28:50,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 05:28:50,882 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 05:28:50,882 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-982671531] 05:28:50,882 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:50,883 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:50,883 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-2139439136] 05:28:50,883 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2000655999] 05:28:50,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:50,883 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] 05:28:50,883 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 05:28:50,883 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from 0 to 1 05:28:50,883 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 05:28:50,883 PM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:50,883 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=1 05:28:50,885 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] with 27000 ms timeout 05:28:50,885 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 05:28:50,885 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 05:28:50,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 05:28:50,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:50,885 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 05:28:50,885 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:50,886 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,886 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-1540346822] 05:28:50,933 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2000655999] 05:28:50,933 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:50,933 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 05:28:50,934 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 05:28:50,934 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 05:28:50,934 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 05:28:50,934 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 05:28:50,935 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 05:28:50,935 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,935 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:50,935 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:50,936 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 05:28:50,936 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:50,936 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 05:28:50,936 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:28:50,937 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 05:28:50,937 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 05:28:50,937 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 05:28:50,937 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,937 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 05:28:50,937 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,938 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:50,938 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:50,939 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:50,939 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:50,946 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:50,947 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:50,947 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:50,947 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:50,950 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:50,954 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 05:28:50,968 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:50,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 05:28:50,969 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 05:28:50,970 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onShutDown 05:28:50,971 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardReplica 05:28:50,971 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,971 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:28:50,971 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:28:50,972 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onShutDown 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onShutDown 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardReplica 05:28:50,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,973 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardReplica 05:28:50,973 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:50,974 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:28:50,974 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:28:50,975 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:50,976 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:51,838 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:51,958 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7418068649050882059] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:51,988 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-7576886986274860974] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:51,990 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:51,990 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:51,992 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:51.995] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:52,7 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onShutDown 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Initiating leader transfer 05:28:52,8 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onShutDown 05:28:52,8 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7f0a3485 05:28:52,8 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7f0a3485 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a36681e 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a36681e 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Attempting to transfer leadership 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,8 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,9 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 05:28:52,9 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardReplica 05:28:52,9 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:52,558 PM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 05:28:52,558 PM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 549.9 ms 05:28:52,558 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 05:28:52,558 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: leader transfer failed - sending PoisonPill 05:28:52,558 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Leadership transfer expired 05:28:52,559 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardReplica 05:28:52,559 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:52,560 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:52,560 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:52,560 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 05:28:52,561 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 05:28:52,561 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 05:28:52,561 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onShutDown 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Initiating leader transfer 05:28:52,561 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onShutDown 05:28:52,561 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Initiating leader transfer 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f6e637a 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f6e637a 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 05:28:52,561 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 05:28:52,561 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 05:28:52,562 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,562 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10cdbfcd 05:28:52,562 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10cdbfcd 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,562 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:52,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 05:28:52,868 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 306.8 ms 05:28:52,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 05:28:52,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 05:28:52,868 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Leadership transfer expired 05:28:52,869 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardReplica 05:28:52,869 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:52,917 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 05:28:52,918 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 356.3 ms 05:28:52,918 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 05:28:52,918 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 05:28:52,918 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Leadership transfer expired 05:28:52,918 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardReplica 05:28:52,919 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Aborting 0 pending queued transactions 05:28:52,919 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:52,919 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:52,921 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:52,921 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:52,925 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:52.928] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:52,958 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:52,969 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4931165103480165097] 05:28:52,970 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:52,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:52,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:52,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:52,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:52,987 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:52,989 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:52,989 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:52,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:52,990 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:52,993 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 05:28:53,0 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:53,3 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,6 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:53,6 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,6 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,7 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,7 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,7 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,8 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:53,9 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e7c2e95 05:28:53,10 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 05:28:53,11 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,11 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,11 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,12 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,12 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,12 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] 05:28:53,12 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,13 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,14 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,14 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,14 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1760125224] 05:28:53,14 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveAllShardReplicas, persistent : true 05:28:53,15 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1760125224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,16 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,16 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveAllShardReplicas, persistent : true 05:28:53,17 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 05:28:53,17 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1081405048 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,17 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1081405048] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,17 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#921416062] 05:28:53,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#921416062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,18 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1548438660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,18 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,20 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:53,33 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@348445e5 05:28:53,35 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,37 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,37 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,37 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:28:53,37 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 05:28:53,37 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,37 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,37 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,39 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-368196784] 05:28:53,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#2004538352 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,39 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 05:28:53,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-368196784] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#2004538352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,39 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,40 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 05:28:53,40 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,40 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,40 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,40 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:53,40 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 05:28:53,40 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,40 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,40 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,41 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,41 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 05:28:53,41 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,41 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,41 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,41 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,41 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1836258029] 05:28:53,42 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,42 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,42 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1836258029] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,42 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,42 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,42 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1365482740] 05:28:53,42 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#2001349292 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,43 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,43 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#2001349292] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,43 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,43 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,43 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,43 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,43 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,43 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,44 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#576502082 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,44 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1307975276] 05:28:53,44 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,42 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#410807201] 05:28:53,44 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:53,44 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 05:28:53,44 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,44 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,45 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,45 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,45 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:28:53,45 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 05:28:53,45 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,45 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,45 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,45 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,45 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,51 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:28:53,90 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2689687245923057650] 05:28:53,90 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:53,95 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:53,95 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:53,101 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:53,101 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:53,110 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,112 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,115 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 05:28:53,131 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:53,136 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:53,147 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,148 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:53,148 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,148 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,149 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a72b1b9 05:28:53,150 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 05:28:53,151 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,151 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,152 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveAllShardReplicas, persistent : true 05:28:53,152 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,153 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,153 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveAllShardReplicas, persistent : true 05:28:53,153 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,153 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#697533426] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,153 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#773091670 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,154 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1857351500] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,154 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1540215442] 05:28:53,154 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#773091670] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,154 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1267857623] 05:28:53,154 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1540215442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,155 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 05:28:53,156 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:53,158 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#552639577 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,159 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:28:53,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 05:28:53,160 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,160 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,162 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:53,162 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 05:28:53,164 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:53,164 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,164 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:53,166 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:53,166 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,166 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,166 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,166 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,167 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 05:28:53,167 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1267857623] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#624366650] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,167 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#552639577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,168 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 05:28:53,168 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@58ac69b4 05:28:53,168 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,168 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,169 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,170 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,170 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#359794780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,170 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1417521438 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,170 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1417521438] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,170 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1499405919] 05:28:53,171 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:53,171 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,171 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,171 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,171 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,171 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,171 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,172 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,172 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-475887986 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:53,172 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1843039982] 05:28:53,172 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:53,172 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 05:28:53,172 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,172 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,172 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,173 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,173 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:28:53,173 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 05:28:53,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,173 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,173 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,173 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,173 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,182 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:53,188 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:53,191 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,192 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,192 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,202 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7874311578695186366] 05:28:53,204 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:28:53,205 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:53,205 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:28:53,209 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:53,209 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:53,215 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,216 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,218 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 05:28:53,224 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:28:53,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,227 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:28:53,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,227 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,230 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:53,231 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@67251593 05:28:53,231 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 05:28:53,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,232 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,232 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,233 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,233 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:53,233 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveAllShardReplicas, persistent : true 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-696602830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-166752720 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,234 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveAllShardReplicas, persistent : true 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-166752720] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1347340916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,234 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#626335937] 05:28:53,234 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#626335937] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,237 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 05:28:53,240 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:28:53,240 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1628293384 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,240 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1839795369] 05:28:53,240 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1340517259] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,240 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,240 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,242 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 05:28:53,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:28:53,242 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 05:28:53,245 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:28:53,245 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1628293384] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,246 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1839795369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,246 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:28:53,246 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,247 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,247 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,247 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,247 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,249 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,249 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,249 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,249 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:53,249 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,249 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,249 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,249 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,251 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,251 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,251 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:28:53,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 05:28:53,251 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 05:28:53,251 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,252 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:53,252 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 05:28:53,253 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:53,254 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 05:28:53,255 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:53,255 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7eaa77f7 05:28:53,255 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:53,255 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,256 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,257 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2074620032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,257 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#535044701] 05:28:53,257 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1499317149 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,257 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,257 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1499317149] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:53,257 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,257 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,258 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,258 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,258 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,258 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,258 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,258 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,259 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2074620032] 05:28:53,259 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,259 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveAllShardReplicas, persistent : true 05:28:53,260 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-594742543 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,260 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1926102204] 05:28:53,260 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 05:28:53,260 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,260 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,261 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,261 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#2025431462] 05:28:53,261 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:28:53,261 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 05:28:53,261 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:28:53,261 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 05:28:53,261 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,261 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,261 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,380 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,381 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 05:28:53,381 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,409 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,410 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 05:28:53,410 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 05:28:53,429 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 05:28:53,429 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 05:28:53,459 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 05:28:53,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 05:28:53,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:53,730 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,730 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 05:28:53,759 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:53,759 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,759 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 05:28:53,799 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:53,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,800 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 05:28:53,859 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:53,859 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,859 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 05:28:53,991 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:28:53,991 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:53,992 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:53,992 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:53,992 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,992 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:53,993 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:53,993 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,993 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:53,993 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,995 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:53,994 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:53,993 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:53,996 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:53,996 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,2 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:54,2 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,3 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,123 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:54,123 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:54,123 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 05:28:54,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,124 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:54,126 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:54,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:54,129 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 05:28:54,129 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,129 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,129 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:54,130 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 05:28:54,130 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,130 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:54,130 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 05:28:54,130 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 05:28:54,130 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:54,130 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 05:28:54,130 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 05:28:54,130 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,130 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,130 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 05:28:54,131 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,131 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,132 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,140 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:54,140 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 05:28:54,140 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 05:28:54,140 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,140 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,140 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:54,140 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:54,140 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 05:28:54,141 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,141 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 05:28:54,141 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,141 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:54,142 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:54,142 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 05:28:54,143 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 05:28:54,143 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 05:28:54,143 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 05:28:54,144 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,144 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,144 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 05:28:54,144 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 05:28:54,144 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 05:28:54,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 05:28:54,144 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,144 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 05:28:54,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] 05:28:54,144 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:54,144 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] 05:28:54,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] 05:28:54,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] 05:28:54,145 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:54,145 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#697533426] 05:28:54,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#697533426] 05:28:54,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] 05:28:54,145 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:54,145 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-696602830] 05:28:54,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-696602830] 05:28:54,145 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:54,145 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 05:28:54,146 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:54,146 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,146 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,146 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,146 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 05:28:54,146 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 05:28:54,146 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,147 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,147 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 05:28:54,147 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:54,147 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,147 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,147 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,147 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,147 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 05:28:54,148 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,148 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,156 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@399a4cdf] 05:28:54,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 05:28:54,159 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-pets-config_testRemoveAllShardReplicas, persistent : true 05:28:54,160 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1111933428 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,160 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#1030510346] 05:28:54,160 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 05:28:54,161 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:54,161 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:54,161 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 05:28:54,161 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 05:28:54,161 PM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:54,161 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 05:28:54,161 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:54,161 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:54,161 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 05:28:54,170 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7322f7a3] 05:28:54,171 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 05:28:54,172 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-2-shard-pets-config_testRemoveAllShardReplicas, persistent : true 05:28:54,173 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#2121936164 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 05:28:54,173 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-208165425] 05:28:54,173 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 05:28:54,174 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:54,174 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:54,174 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 05:28:54,174 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 05:28:54,174 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:54,174 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 05:28:54,174 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:54,174 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:54,175 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 05:28:54,182 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@74521c87] 05:28:54,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 05:28:54,184 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-pets-config_testRemoveAllShardReplicas, persistent : true 05:28:54,185 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-685161692] 05:28:54,185 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-1391867229 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 05:28:54,185 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 05:28:54,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] 05:28:54,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 05:28:54,185 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1876184035] 05:28:54,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1876184035] 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 05:28:54,186 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 05:28:54,186 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 05:28:54,186 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 05:28:54,186 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 05:28:54,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] 05:28:54,186 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 05:28:54,186 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-264601782] 05:28:54,186 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-264601782] 05:28:54,187 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] 05:28:54,187 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 05:28:54,187 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-830880386] 05:28:54,187 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-830880386] 05:28:54,188 PM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 05:28:54,189 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 05:28:54,189 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 05:28:54,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 05:28:54,189 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 05:28:54,189 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 05:28:54,189 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 05:28:54,190 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 05:28:54,190 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 05:28:54,190 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 05:28:54,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 05:28:54,195 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 05:28:54,195 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,195 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,196 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'}, state: Idle 05:28:54,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]] 05:28:54,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=434783c0-c542-49b0-b8f2-f43c6543f963}, logIndex: 0, isSendImmediate: true 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'} 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'}, state: Idle 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]] 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ad2afca0-0deb-44d6-b46e-bf0540fb21b4}, logIndex: 0, isSendImmediate: true 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,198 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'} 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'}, state: Idle 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]] 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=de67b552-2628-4294-af6a-730bcdcaad18}, logIndex: 0, isSendImmediate: true 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'} 05:28:54,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 05:28:54,200 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 05:28:54,200 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:28:54,200 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,200 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 05:28:54,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 05:28:54,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 05:28:54,201 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:28:54,201 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 05:28:54,201 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 05:28:54,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 05:28:54,201 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,201 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:28:54,201 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 05:28:54,201 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 05:28:54,201 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 05:28:54,201 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:54,201 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:54,201 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,201 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,202 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, votingState=VOTING]] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, votingState=VOTING]] 05:28:54,202 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Log size is now 1 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Log size is now 1 05:28:54,202 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, votingState=VOTING]] 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 05:28:54,202 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:54,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,203 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 05:28:54,203 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,203 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 05:28:54,203 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 05:28:54,203 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 05:28:54,203 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 05:28:54,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 05:28:54,203 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:54,203 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:54,204 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,204 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:54,204 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:54,205 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,205 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1347340916] with 27000 ms timeout 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,205 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 05:28:54,205 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, cars] 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onShutDown 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,205 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:28:54,205 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#2025431462] with 27000 ms timeout 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveAllShardReplicas 05:28:54,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onShutDown 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,206 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1347340916] 05:28:54,206 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveAllShardReplicas 05:28:54,206 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-696602830] with 27000 ms timeout 05:28:54,206 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 05:28:54,206 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 05:28:54,206 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#2025431462] 05:28:54,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 05:28:54,206 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onShutDown 05:28:54,206 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 05:28:54,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:54,207 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:54,207 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveAllShardReplicas 05:28:54,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:28:54,207 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 05:28:54,207 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,207 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-696602830] 05:28:54,222 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:28:54,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 05:28:54,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,270 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 05:28:54,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,270 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,271 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,271 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 05:28:54,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 05:28:54,272 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 05:28:54,272 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 05:28:54,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,274 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,274 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 05:28:54,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 05:28:54,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,274 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 05:28:54,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 05:28:54,275 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 05:28:54,275 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 05:28:54,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,275 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'}, state: Idle 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]] 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bded3f81-63bb-4a06-b839-020687292c9a}, logIndex: 0, isSendImmediate: true 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'} 05:28:54,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 05:28:54,279 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:54,279 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,279 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, votingState=VOTING]] 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 05:28:54,279 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:54,279 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,280 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:54,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2074620032] with 27000 ms timeout 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,281 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 05:28:54,281 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:54,281 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,282 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 05:28:54,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,282 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-2074620032] 05:28:54,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:28:54,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 05:28:54,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 05:28:54,550 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,550 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 05:28:54,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-pets-config_testRemoveAllShardReplicas 05:28:54,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-pets-config_testRemoveAllShardReplicas 05:28:54,551 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:54,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 05:28:54,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 05:28:54,551 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 05:28:54,552 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 05:28:54,553 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,553 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=1, voteGranted=true], my term is 1 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=1, voteGranted=true], current voteCount: 1 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 05:28:54,554 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 05:28:54,554 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,555 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 05:28:54,555 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,555 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,555 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,556 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,556 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 05:28:54,556 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: getLeaderAddress leaderId = member-1-shard-pets-config_testRemoveAllShardReplicas peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:54,556 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,556 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 05:28:54,556 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,556 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 05:28:54,556 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 05:28:54,557 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 05:28:54,557 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 05:28:54,558 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'}, state: Idle 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]] 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=0c40c221-0d37-41bb-9163-891e42d39151}, logIndex: 0, isSendImmediate: true 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testRemoveAllShardReplicas 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'} 05:28:54,560 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,560 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 05:28:54,560 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 05:28:54,560 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 05:28:54,560 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testRemoveAllShardReplicas, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, votingState=VOTING]] 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Log size is now 1 05:28:54,561 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:54,562 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] 05:28:54,562 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 05:28:54,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:54,562 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 05:28:54,562 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] 05:28:54,562 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1142011505] 05:28:54,562 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] 05:28:54,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:28:54,562 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1760125224] 05:28:54,562 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1760125224] 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,563 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#674559691] 05:28:54,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 05:28:54,563 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1876184035] 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-830880386] with 27000 ms timeout 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onShutDown 05:28:54,564 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#1876184035] 05:28:54,564 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:54,564 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-pets-config_testRemoveAllShardReplicas 05:28:54,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] 05:28:54,565 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:28:54,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:54,565 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#697533426] 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#697533426] 05:28:54,565 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-830880386] 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] 05:28:54,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:28:54,565 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1857351500] 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1857351500] 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#1666853152] 05:28:54,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 05:28:54,565 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-264601782] 05:28:54,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-264601782] 05:28:54,566 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 05:28:54,566 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 05:28:54,566 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1372452583] 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 05:28:54,566 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 05:28:54,566 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:28:54,566 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:28:54,567 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:54,567 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 05:28:54,567 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:28:54,567 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:28:54,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:54,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:54,573 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:54,573 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:54,574 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:54,574 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:54,578 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:54,591 PM [cluster-test-shard-dispatcher-30] [WARN] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 309, lastApplied : 0, commitIndex : 0 05:28:54,593 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:54,594 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 05:28:54,595 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,595 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onShutDown 05:28:54,595 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,595 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 05:28:54,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onShutDown 05:28:54,595 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onShutDown 05:28:54,595 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testRemoveAllShardReplicas 05:28:54,595 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveAllShardReplicas 05:28:54,595 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,596 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveAllShardReplicas 05:28:54,596 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,596 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:28:54,596 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:28:54,596 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:54,597 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,597 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,597 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 05:28:54,597 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:28:54,597 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onShutDown 05:28:54,597 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 05:28:54,597 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,598 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveAllShardReplicas 05:28:54,598 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:54,598 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:28:54,598 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:28:54,599 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:54,599 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:55,11 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:28:55,590 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [7874311578695186366] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:55,619 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2689687245923057650] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:28:55,624 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:55,625 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:28:55,628 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:55.629] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:55,638 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onShutDown 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Initiating leader transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onShutDown 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Initiating leader transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onShutDown 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Initiating leader transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e256d32 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4fb0e39b 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e256d32 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4fb0e39b 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35b54584 05:28:55,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35b54584 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 05:28:55,639 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 05:28:55,639 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 05:28:56,50 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 05:28:56,50 PM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 410.9 ms 05:28:56,50 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 05:28:56,50 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 05:28:56,50 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 05:28:56,51 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveAllShardReplicas 05:28:56,51 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:56,59 PM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 05:28:56,59 PM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 420.1 ms 05:28:56,59 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 05:28:56,59 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 05:28:56,59 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 05:28:56,60 PM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveAllShardReplicas 05:28:56,60 PM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:56,149 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 05:28:56,149 PM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 510.3 ms 05:28:56,149 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 05:28:56,149 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 05:28:56,149 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 05:28:56,150 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testRemoveAllShardReplicas 05:28:56,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:56,151 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:56,151 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:56,151 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 05:28:56,152 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:56,152 PM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:56,152 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onShutDown 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Initiating leader transfer 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Initiating leader transfer 05:28:56,152 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12013a1e 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ab26409 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12013a1e 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ab26409 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 05:28:56,152 PM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 05:28:56,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:56,152 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:56,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 05:28:56,510 PM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 358.0 ms 05:28:56,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 05:28:56,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 05:28:56,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 05:28:56,511 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 05:28:56,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:56,699 PM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 05:28:56,700 PM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 547.6 ms 05:28:56,700 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 05:28:56,700 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 05:28:56,700 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 05:28:56,701 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveAllShardReplicas 05:28:56,701 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Aborting 0 pending queued transactions 05:28:56,701 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:56,701 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:56,703 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:56,703 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:56,706 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:28:56.709] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:56,732 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:28:56,748 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8870584343263003523] 05:28:56,748 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:56,750 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:56,752 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:56,752 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:56,752 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:56,765 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:56,765 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:56,766 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,766 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:56,768 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,770 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 05:28:56,774 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:56,774 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#274295507] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,776 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:56,776 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,777 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:56,777 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,780 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 05:28:56,781 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 05:28:56,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@efa5711 05:28:56,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:56,782 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,783 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,783 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,783 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1217540987] 05:28:56,783 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,783 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplicaFailures, persistent : true 05:28:56,784 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplicaFailures with journal batch size 1 05:28:56,784 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1217540987] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,784 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-690070946 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,785 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-690070946] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,785 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#-527161103] 05:28:56,785 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#-527161103] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,785 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,785 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:56,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:56,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,786 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 05:28:56,786 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 05:28:56,786 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 05:28:56,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 05:28:56,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 05:28:56,787 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:56,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 05:28:56,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 05:28:56,789 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 05:28:56,791 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:56,791 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 05:28:56,791 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 05:28:56,792 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderPayloadVersion=13] 05:28:56,792 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 05:28:56,792 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 05:28:56,792 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#458866292] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,793 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:56,793 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,793 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:56,793 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,797 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 05:28:56,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@520dff10 05:28:56,798 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 05:28:56,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:56,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,799 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,799 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2081485613] 05:28:56,799 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplicaFailures, persistent : true 05:28:56,800 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplicaFailures with journal batch size 1 05:28:56,800 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#358615008] 05:28:56,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#30479713 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2081485613] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#30479713] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,801 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#358615008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:56,802 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:56,802 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 05:28:56,802 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 05:28:56,802 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 05:28:56,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 05:28:56,803 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 05:28:56,803 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard people 05:28:56,803 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 05:28:56,803 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderPayloadVersion=13] 05:28:56,803 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 05:28:56,803 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 05:28:56,803 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 05:28:56,803 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 05:28:56,803 PM [cluster-test-akka.actor.default-dispatcher-18] [ERROR] ClusterAdminRpcService - Failed to add replica for shard people java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1188) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:241) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 05:28:56,804 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 05:28:56,804 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,804 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 05:28:56,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onShutDown 05:28:56,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@632dbb20 05:28:56,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@632dbb20 05:28:56,804 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplicaFailures 05:28:56,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Aborting 0 pending queued transactions 05:28:56,804 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:28:56,805 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:28:56,805 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 05:28:56,805 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,805 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 05:28:56,805 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onShutDown 05:28:56,805 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1d1e276b 05:28:56,805 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1d1e276b 05:28:56,805 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#274295507] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#274295507] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,805 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplicaFailures 05:28:56,805 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Aborting 0 pending queued transactions 05:28:56,805 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#274295507] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#274295507] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,806 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:28:56,806 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:28:56,807 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:28:56,807 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:28:56,808 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:28:56,834 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:56,847 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5103006086431565391] 05:28:56,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:28:56,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:28:56,851 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:28:56,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:56,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:56,864 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,866 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:28:56,866 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:28:56,866 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:28:56,867 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,869 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 05:28:56,872 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-220530734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,873 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:28:56,879 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:28:56,879 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 05:28:56,879 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:56,879 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:56,903 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 05:28:56,904 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2493190d 05:28:56,904 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 05:28:56,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:56,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 05:28:56,906 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:56,907 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:56,907 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934] 05:28:56,907 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplica, persistent : true 05:28:56,907 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-182506183 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 05:28:56,908 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#1175622604] 05:28:56,908 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-182506183] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#1175622604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,908 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplica with journal batch size 1 05:28:56,909 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:56,909 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:56,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:28:56,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 05:28:56,910 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplica 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,910 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 05:28:56,914 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Received ElectionTimeout 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Election: Leader has following peers: [] 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 05:28:56,914 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 05:28:56,914 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 05:28:56,914 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 05:28:56,915 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 05:28:56,915 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 05:28:56,956 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-220530734] 05:28:56,956 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:28:56,956 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934] 05:28:56,956 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,959 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:56,962 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 05:28:56,964 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#196273444] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,965 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:28:56,968 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:28:56,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 05:28:56,968 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:56,968 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:56,970 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 05:28:56,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@570edec5 05:28:56,971 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 05:28:56,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:56,971 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 05:28:56,972 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:56,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:56,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:28:56,972 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplica, persistent : true 05:28:56,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,973 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#-290395552 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 05:28:56,973 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-1837546858] 05:28:56,973 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#-290395552] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,973 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-1837546858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:56,973 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplica with journal batch size 1 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:56,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:28:56,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplica 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 05:28:56,974 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 05:28:56,974 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Received ElectionTimeout 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Election: Leader has following peers: [] 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:28:56,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:28:56,975 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 05:28:56,975 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 05:28:56,975 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 05:28:56,975 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 05:28:56,975 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 05:28:56,975 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:57,21 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#196273444] 05:28:57,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:28:57,22 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:28:57,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:28:57,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1473859953625752528] 05:28:57,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:28:57,50 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:57,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:28:57,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:57,51 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:57,57 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,58 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,63 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 05:28:57,68 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:28:57,68 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1098851529] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,72 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:28:57,74 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 05:28:57,75 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 05:28:57,76 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,78 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,78 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@ef99d05 05:28:57,79 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:57,80 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 05:28:57,84 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 05:28:57,84 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,85 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:28:57,87 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 05:28:57,89 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:28:57,89 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4fc4973e 05:28:57,89 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:57,104 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:28:57,106 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:57,107 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:57,107 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:57,107 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:57,107 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 05:28:57,869 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:28:57,870 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:57,870 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:28:57,870 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:57,870 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:57,870 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 05:28:57,870 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:57,871 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 05:28:57,871 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:57,889 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:57,889 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:57,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:57,890 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 05:28:57,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:57,892 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934, primaryVersion=13] 05:28:57,895 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddShardReplica, persistent : true 05:28:57,895 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,896 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-1162620006 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 05:28:57,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#1654168674] 05:28:57,896 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-1162620006] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,896 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#1654168674] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,896 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddShardReplica with journal batch size 1 05:28:57,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934 for shard member-2-shard-cars-config_testAddShardReplica 05:28:57,898 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:57,898 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:57,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:28:57,898 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 05:28:57,899 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:57,899 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 05:28:57,899 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 05:28:57,899 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 05:28:57,899 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 1 05:28:57,899 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:57,900 PM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-config_testAddShardReplica 05:28:57,900 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 05:28:57,900 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@20649c96, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=null] 05:28:57,900 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:57,900 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddShardReplica: 229 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-2-shard-cars-config_testAddShardReplica), Chunk: 1/1 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 05:28:57,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 05:28:57,902 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:57,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 05:28:57,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:57,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:57,902 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 05:28:57,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:57,903 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 05:28:57,903 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@70cad283, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:57,904 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to -1 05:28:57,904 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#264249560] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 05:28:57,905 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#264249560] 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 493.6 μs 05:28:57,905 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 05:28:57,905 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 05:28:57,905 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#264249560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,905 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 123.1 μs 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: SnapshotComplete received 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a snapshot response 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a journal response 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:57,906 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddShardReplica] 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9c56e97b-305c-4a08-945f-551e0c08933e}, logIndex: 0, isSendImmediate: true 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddShardReplica 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:28:57,907 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 05:28:57,908 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 05:28:57,909 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 05:28:57,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:57,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:57,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934 05:28:57,910 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 05:28:57,910 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 05:28:57,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 0 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:28:57,911 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:57,911 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1098851529] 05:28:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:57,911 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 05:28:57,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 05:28:57,911 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:28:57,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 05:28:57,911 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:28:57,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 05:28:57,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 0 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 0 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:28:57,912 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 05:28:57,912 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 05:28:57,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:57,913 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 05:28:57,913 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:28:57,913 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 05:28:57,914 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:28:57,914 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 05:28:57,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:28:57,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974, primaryVersion=13] 05:28:57,916 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974 for shard member-2-shard-cars-oper_testAddShardReplica 05:28:57,917 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddShardReplica, persistent : true 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 05:28:57,917 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 1 05:28:57,917 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1567713034] 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:28:57,917 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddShardReplica 05:28:57,917 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1364383357 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 05:28:57,917 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1364383357] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#1567713034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,918 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddShardReplica with journal batch size 1 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ed96849, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=null] 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddShardReplica: 229 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica), Chunk: 1/1 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 05:28:57,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:28:57,918 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 05:28:57,918 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 05:28:57,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:28:57,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 05:28:57,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:28:57,919 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 05:28:57,919 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 05:28:57,919 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 05:28:57,919 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 05:28:57,919 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 05:28:57,919 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 05:28:57,919 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 05:28:57,919 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:28:57,919 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1948ab7e, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:28:57,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to -1 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 05:28:57,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-464006300] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 05:28:57,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-464006300] 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 302.1 μs 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 05:28:57,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: SnapshotComplete received 05:28:57,921 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-464006300] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,921 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 145.1 μs 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a journal response 05:28:57,921 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a snapshot response 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddShardReplica] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1737f8ee-d9fb-4c8a-bd2b-c050386b8688}, logIndex: 0, isSendImmediate: true 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddShardReplica 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:28:57,922 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 05:28:57,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:28:57,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:28:57,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:28:57,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:28:57,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:28:57,924 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:28:57,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974 05:28:57,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 05:28:57,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 05:28:57,924 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 05:28:57,925 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:28:57,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 05:28:57,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] 05:28:57,925 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:28:57,925 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:28:57,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 05:28:57,925 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:28:57,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:28:57,925 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 05:28:57,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:28:57,925 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:57,926 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 0 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 05:28:57,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 0 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 0 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:28:57,929 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 05:28:57,926 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:57,932 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 05:28:57,932 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 05:28:57,938 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:28:57,951 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2451285166125770654] 05:28:57,952 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:28:57,955 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:28:57,955 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:28:57,956 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:28:57,956 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:28:57,972 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,974 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,976 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 05:28:57,980 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:28:57,983 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#866454624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,989 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:28:57,989 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 05:28:57,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4d8adcba 05:28:57,990 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 05:28:57,990 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:57,990 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,992 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:28:57,994 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 05:28:57,995 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:28:57,995 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#438068864] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:28:57,998 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:28:58,1 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 05:28:58,2 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:28:58,3 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 05:28:58,3 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4772b7ca 05:28:58,4 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:28:58,4 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:28:58,9 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:58,9 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:28:58,9 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:58,9 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:28:58,9 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:28:58,9 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:58,9 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:58,9 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:28:58,62 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 05:28:58,220 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 05:28:58,550 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 05:28:58,890 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:28:58,890 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:58,890 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:28:58,890 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:28:58,890 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 05:28:58,890 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:58,890 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 05:28:58,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:28:58,891 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:59,909 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:28:59,914 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:59,914 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:59,914 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 05:28:59,914 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:28:59,914 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 05:28:59,914 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:28:59,914 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 05:28:59,914 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:59,997 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:59,998 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 05:28:59,997 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:28:59,998 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:29:00,9 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:29:00,9 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 05:29:00,10 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,10 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 05:29:00,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934, primaryVersion=13] 05:29:00,13 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934 for shard member-3-shard-cars-config_testAddShardReplica 05:29:00,14 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 05:29:00,14 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1799338867] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,14 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-2104092738 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 05:29:00,15 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-2104092738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,15 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1084622380] 05:29:00,15 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 05:29:00,15 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1084622380] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,15 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true], state: Idle 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 3 05:29:00,16 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:00,16 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:29:00,16 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-config_testAddShardReplica 05:29:00,16 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 05:29:00,16 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:00,16 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 05:29:00,16 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 05:29:00,16 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 05:29:00,16 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@32372d41, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:29:00,16 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testAddShardReplica: 229 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica), Chunk: 1/1 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 3 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 05:29:00,17 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 05:29:00,18 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:00,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 05:29:00,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:29:00,19 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6126894a, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,19 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:29:00,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 05:29:00,19 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:29:00,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 05:29:00,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 05:29:00,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#886153630] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 365.3 μs 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotComplete received 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a journal response 05:29:00,20 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#886153630] 05:29:00,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a snapshot response 05:29:00,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:29:00,21 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#886153630] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,21 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 67.18 μs 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testAddShardReplica] 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=af21bc63-be0d-4aed-896d-bf05f61cc94f}, logIndex: 1, isSendImmediate: true 05:29:00,21 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testAddShardReplica 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testAddShardReplica 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica, votingMember=true] 05:29:00,22 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 05:29:00,23 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:00,23 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 05:29:00,24 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 1 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 05:29:00,24 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 05:29:00,24 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:29:00,24 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 05:29:00,24 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 05:29:00,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 05:29:00,24 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:00,25 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,25 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 05:29:00,25 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#866454624] 05:29:00,25 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 05:29:00,25 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:29:00,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,26 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 05:29:00,26 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1799338867] 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 05:29:00,26 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1799338867] 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 05:29:00,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,26 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#438068864] 05:29:00,26 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:00,26 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 05:29:00,26 PM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 05:29:00,26 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 05:29:00,27 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,27 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 05:29:00,28 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,28 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974, primaryVersion=13] 05:29:00,29 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974 for shard member-3-shard-cars-oper_testAddShardReplica 05:29:00,30 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-cars-oper_testAddShardReplica, persistent : true 05:29:00,31 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testAddShardReplica with journal batch size 1 05:29:00,32 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1565309269] 05:29:00,32 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#311109484 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true], state: Idle 05:29:00,32 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#305861737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 05:29:00,32 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#311109484] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 3 05:29:00,32 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-1565309269] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-oper_testAddShardReplica 05:29:00,32 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4363f511, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testAddShardReplica: 229 05:29:00,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-3-shard-cars-oper_testAddShardReplica), Chunk: 1/1 05:29:00,33 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:00,33 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:00,33 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 3 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 05:29:00,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:00,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 05:29:00,33 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 05:29:00,33 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 05:29:00,33 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-oper_testAddShardReplica 05:29:00,34 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 05:29:00,34 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 05:29:00,34 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 05:29:00,34 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,34 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 05:29:00,34 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 05:29:00,35 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 05:29:00,35 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@70cf7892, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,35 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 05:29:00,35 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 05:29:00,35 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#519018187] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 05:29:00,36 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#519018187] 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 259.8 μs 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: SnapshotComplete received 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a journal response 05:29:00,36 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a snapshot response 05:29:00,37 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#519018187] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,36 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testAddShardReplica] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 05:29:00,37 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 285.5 μs 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8a0f8f86-1cc9-49e1-9747-a93c7c369b9d}, logIndex: 1, isSendImmediate: true 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testAddShardReplica 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testAddShardReplica 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:29:00,37 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica, votingMember=true] 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,38 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 05:29:00,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 1 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 05:29:00,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974 05:29:00,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:29:00,39 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 05:29:00,39 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,39 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testAddShardReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:29:00,40 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 05:29:00,40 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,40 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#438068864] 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 05:29:00,40 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 05:29:00,40 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 05:29:00,40 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#305861737] 05:29:00,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 05:29:00,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 05:29:00,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 05:29:00,41 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,41 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#305861737] 05:29:00,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 05:29:00,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 05:29:00,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 05:29:00,42 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1098851529] 05:29:00,42 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 05:29:00,42 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:29:00,42 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 05:29:00,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:29:00,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] 05:29:00,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:29:00,43 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:29:00,43 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:29:00,82 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1098851529] 05:29:00,82 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,82 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:00,83 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:00,103 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 05:29:00,114 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=90052, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 05:29:00,120 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 05:29:00,122 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 05:29:00,143 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 21.73 ms 05:29:00,146 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,162 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,162 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 05:29:00,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 05:29:00,166 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 05:29:00,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 05:29:00,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 05:29:00,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 05:29:00,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 05:29:00,171 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 05:29:00,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 05:29:00,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 05:29:00,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 05:29:00,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 05:29:00,174 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 05:29:00,175 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:29:00,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testAddShardReplica 05:29:00,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 05:29:00,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:29:00,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testAddShardReplica 05:29:00,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 05:29:00,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 05:29:00,185 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 05:29:00,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:00,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 05:29:00,185 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,186 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 05:29:00,186 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,186 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 05:29:00,186 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 05:29:00,187 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 05:29:00,187 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:29:00,187 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 2 05:29:00,187 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,187 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 05:29:00,188 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,189 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@16806315}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,189 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,190 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,191 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 05:29:00,191 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 05:29:00,192 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#2145343288] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 05:29:00,192 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#2145343288] 05:29:00,193 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:29:00,193 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#2145343288] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,193 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 214.0 μs 05:29:00,201 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#866454624] 05:29:00,202 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,202 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:00,202 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:00,204 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,205 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:29:00,205 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 05:29:00,205 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:29:00,205 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 05:29:00,206 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 05:29:00,207 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,208 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@14cd2ef}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,208 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,209 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,209 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:29:00,209 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,209 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 134.2 μs 05:29:00,209 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2f1b7674}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,209 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:29:00,209 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 88.56 μs 05:29:00,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,211 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testAddShardReplica 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-config_testAddShardReplica 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 05:29:00,213 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=7760, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 05:29:00,216 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 05:29:00,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,217 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 05:29:00,217 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 05:29:00,217 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,218 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.027 ms 05:29:00,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 05:29:00,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:29:00,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 3 05:29:00,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 05:29:00,218 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,219 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f015c96a810@2478560b 05:29:00,219 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 05:29:00,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:00,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 05:29:00,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 05:29:00,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,226 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 05:29:00,226 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,227 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,233 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#438068864] 05:29:00,233 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,233 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 05:29:00,234 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 05:29:00,235 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 05:29:00,235 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=16370, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 05:29:00,236 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 05:29:00,236 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 05:29:00,236 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 573.0 μs 05:29:00,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 05:29:00,238 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testAddShardReplica 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testAddShardReplica 05:29:00,239 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 05:29:00,240 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 05:29:00,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,241 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,241 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 05:29:00,241 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 2 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 05:29:00,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@452ba1c3}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 05:29:00,242 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1402726026] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 05:29:00,242 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1402726026] 05:29:00,242 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 05:29:00,243 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 79.31 μs 05:29:00,243 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1402726026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,243 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] 05:29:00,243 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,243 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 05:29:00,243 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 05:29:00,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:29:00,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 05:29:00,245 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 05:29:00,246 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,246 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 05:29:00,246 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,246 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 05:29:00,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@592f8d3b}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,247 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,247 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,247 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 05:29:00,247 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 119.6 μs 05:29:00,247 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,247 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6611c0e9}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 05:29:00,247 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,248 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 05:29:00,248 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 60.95 μs 05:29:00,248 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 05:29:00,249 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=11290, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 05:29:00,250 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 05:29:00,250 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 05:29:00,251 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 442.5 μs 05:29:00,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,254 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-220530734] 05:29:00,256 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:00,256 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934] 05:29:00,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934] 05:29:00,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#196273444] 05:29:00,257 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:00,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,358 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,409 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,511 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,561 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,612 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,663 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,713 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,764 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testAddShardReplica 05:29:00,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 05:29:00,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-oper_testAddShardReplica 05:29:00,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 05:29:00,789 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,789 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 05:29:00,790 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:00,791 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,791 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 3 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f015c96a810@8bfdae0 05:29:00,791 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 05:29:00,792 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:00,793 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,793 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 05:29:00,793 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 05:29:00,793 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 05:29:00,794 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 05:29:00,794 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:00,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 05:29:00,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] 05:29:00,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#-1098851529] 05:29:00,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:00,816 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:29:00,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#1670868185] 05:29:00,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#-1863517936] 05:29:00,816 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:29:00,816 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:29:00,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-2094244349] 05:29:00,816 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#866454624] 05:29:00,816 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:00,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1799338867] 05:29:00,817 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1799338867] 05:29:00,817 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica#438068864] 05:29:00,817 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:00,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#305861737] 05:29:00,817 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#305861737] 05:29:00,817 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 05:29:00,818 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 05:29:00,818 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 05:29:00,818 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 05:29:00,818 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 05:29:00,818 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 05:29:00,818 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node [akka://cluster-test@127.0.0.1:2557] as [Down] 05:29:00,819 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:00,819 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:00,819 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 05:29:00,820 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 05:29:00,820 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 05:29:00,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onShutDown 05:29:00,821 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testAddShardReplica 05:29:00,821 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 05:29:00,821 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:00,821 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:00,822 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [akka://cluster-test@127.0.0.1:2558] 05:29:00,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 05:29:00,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Shutting down... 05:29:00,822 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Successfully shut down 05:29:00,825 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:00,825 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:00,829 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 05:29:00,833 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:00,833 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:00,833 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:00,833 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:00,835 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:00,859 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:29:00,868 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8665394338344847460] 05:29:00,868 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:00,870 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:00,870 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:00,873 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:00,873 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:00,885 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:00,888 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:00,892 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 05:29:00,903 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1900250264] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,904 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:00,905 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: onSnapshotOffer: ShardManagerSnapshot [ShardList = [cars] ] 05:29:00,905 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 05:29:00,907 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:00,907 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 05:29:00,910 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@486fc9bd 05:29:00,910 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 05:29:00,911 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:00,911 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 05:29:00,911 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1900250264] 05:29:00,911 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:00,911 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 05:29:00,912 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 05:29:00,912 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 05:29:00,912 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1593541978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,913 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#163381587 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 05:29:00,913 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#163381587] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,913 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#1272703588] 05:29:00,913 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#1272703588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#-1038574146] 05:29:00,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotOffer called. 05:29:00,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 05:29:00,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 05:29:00,913 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 05:29:00,918 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 05:29:00,920 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#874732083] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 05:29:00,920 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 1.626 ms 05:29:00,920 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#874732083] 05:29:00,920 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 05:29:00,920 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 6.300 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 05:29:00,920 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#874732083] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:00,920 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 05:29:00,922 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 2.061 ms 05:29:00,921 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:00,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 1, size: 267 05:29:00,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 05:29:00,922 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 1 to 1 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 2, size: 235 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 2 to 2 05:29:00,923 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 05:29:00,928 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:00,932 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@39fdef42}, purged=MutableUnsignedLongSet{size=0}} 05:29:00,932 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,932 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,932 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 05:29:00,932 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 3, size: 56 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 3 to 3 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:00,933 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 19.33 ms - Switching actor to Follower - last log index = 3, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 3 05:29:00,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:00,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 05:29:00,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 05:29:00,933 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1593541978] 05:29:00,933 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1593541978] 05:29:00,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:00,936 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 05:29:00,936 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 05:29:00,936 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener akka://cluster-test/user/shardmanager-config_testAddShardReplica 05:29:00,936 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-config_testAddShardReplica#1900250264] 05:29:00,937 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 05:29:00,937 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:00,937 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 05:29:01,847 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [2451285166125770654] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:01,849 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [2451285166125770654] but current is [8665394338344847460]. Reason: ActorSystem terminated 05:29:01,894 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRefProvider$$anon$2 - akkaDeadLetter Message [akka.cluster.ddata.Replicator$Internal$Status] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [1473859953625752528]. This system with UID [8665394338344847460] was probably restarted. Messages will be accepted when new handshake has been completed.. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:01,913 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRefProvider$$anon$2 - akkaDeadLetter Message [akka.cluster.ddata.Replicator$Internal$Status] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [-5103006086431565391]. This system with UID [8665394338344847460] was probably restarted. Messages will be accepted when new handshake has been completed.. [7] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:01,914 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 05:29:01,914 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testAddShardReplica peerAddress = null 05:29:01,914 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 05:29:01,914 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 05:29:01,914 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 05:29:01,915 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 05:29:01,915 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:01,915 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 05:29:01,915 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:01,916 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-1358639974] to Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:01,917 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:01,917 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:02,381 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - New leader address: akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:02,381 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:02,908 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:02,919 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#210155330]] intended for system uid [2451285166125770654], self uid is [8665394338344847460] 05:29:03,119 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1657014310]] intended for system uid [2451285166125770654], self uid is [8665394338344847460] 05:29:03,929 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:04,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:05,10 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 05:29:05,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:05,11 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:05,139 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1657014310]] intended for system uid [2451285166125770654], self uid is [8665394338344847460] 05:29:05,198 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Down)]. 05:29:05,198 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:05,198 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:05,968 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 05:29:06,30 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2557] 05:29:06,31 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:06,31 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:06,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [2451285166125770654] but current is [8665394338344847460]. Reason: Cluster member removed, previous status [Down] 05:29:06,35 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:06,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:06,36 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - akkaQuarantine Quarantine of [akka://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [2451285166125770654] but current is [8665394338344847460]. Reason: Cluster member removed, previous status [Down] 05:29:06,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:06,991 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:06,994 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:29:06,994 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:29:06,994 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 05:29:06,994 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 05:29:06,998 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6771368288, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 05:29:07,0 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=15771, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 05:29:07,1 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 05:29:07,1 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 05:29:07,1 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#898823934], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 483.9 μs 05:29:07,2 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 05:29:07,5 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 05:29:07,5 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 05:29:07,5 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 05:29:07,5 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 05:29:07,5 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 05:29:07,5 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 05:29:07,6 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:07,6 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:07,7 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:07,7 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:07,13 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:07,13 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:07,13 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:07,13 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:07,15 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:07,27 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 05:29:07,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 05:29:07,27 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 05:29:07,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onShutDown 05:29:07,28 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddShardReplica 05:29:07,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 05:29:07,28 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:07,28 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:07,28 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 05:29:07,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 05:29:07,29 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 05:29:07,29 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onShutDown 05:29:07,29 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddShardReplica 05:29:07,29 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 05:29:07,30 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:07,30 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:07,31 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:07,31 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:08,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [8665394338344847460] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:08,51 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:08,51 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:08,51 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1473859953625752528] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:08,53 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:08.055] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:08,67 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 05:29:08,68 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 05:29:08,68 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 05:29:08,68 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onShutDown 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating leader transfer 05:29:08,69 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b0d54af 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b0d54af 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Attempting to transfer leadership 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:08,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:08,69 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to akka://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to akka://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 05:29:08,70 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 05:29:09,88 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:12,149 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2559, Up)]. 05:29:12,149 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:12,149 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:12,156 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:12,304 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:14,189 PM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 05:29:14,190 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:14,190 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:14,545 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:15,210 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:17,98 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 05:29:17,98 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.029 s 05:29:17,98 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 05:29:17,109 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: leader transfer failed - sending PoisonPill 05:29:17,109 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Leadership transfer expired 05:29:17,113 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplica 05:29:17,113 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Aborting 0 pending queued transactions 05:29:17,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:17,114 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:17,114 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 05:29:17,115 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 05:29:17,115 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onShutDown 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating leader transfer 05:29:17,115 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e3b1aa7 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e3b1aa7 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Attempting to transfer leadership 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:17,115 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:17,116 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 05:29:17,116 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 05:29:17,116 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-oper_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica] 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 314.3 ms 05:29:17,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 05:29:17,430 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: leader transfer failed - sending PoisonPill 05:29:17,430 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:17,430 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 05:29:17,430 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 05:29:17,430 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplica 05:29:17,431 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Aborting 0 pending queued transactions 05:29:17,431 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:17,431 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:17,432 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:17,433 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:17,436 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:17.438] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:17,457 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:29:17,468 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8651485197684462251] 05:29:17,469 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:17,471 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:17,471 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:17,473 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:17,473 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:17,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:17,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:17,485 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,485 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:17,486 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,489 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,492 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#149556980] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,495 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:17,496 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:17,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,496 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,497 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,497 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,497 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,501 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@59b6d185 05:29:17,502 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,503 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,503 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] 05:29:17,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,504 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,504 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,504 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,505 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,505 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,505 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,505 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,506 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,506 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,507 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1286593785] 05:29:17,507 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1616413824 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,507 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1254217081] 05:29:17,507 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:17,508 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1474054797 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,508 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1808326840] 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,510 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:17,510 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:17,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,511 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,511 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,511 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,511 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,511 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1808326840] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-222806884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1286593785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1616413824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1254217081] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,511 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1474054797] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,511 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:17,511 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,511 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,511 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,512 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,512 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,512 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,513 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4fe26445 05:29:17,513 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-619863210] 05:29:17,514 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,515 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-619863210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-163037568] 05:29:17,515 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,515 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-163037568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,515 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#931836239 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,514 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,516 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,516 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,516 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,516 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,516 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,516 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1191705332 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,516 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-107226598] 05:29:17,516 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,516 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,516 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,516 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,516 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-215822307] 05:29:17,517 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:17,517 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,517 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,517 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,517 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,517 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,517 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:17,517 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,517 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,518 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,518 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,519 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:17,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5810478384434101979] 05:29:17,528 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:29:17,531 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:17,531 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:29:17,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:17,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:17,545 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,546 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,548 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,570 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:29:17,573 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1776439863] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,576 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:29:17,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,577 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,579 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,580 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@42b36ad2 05:29:17,580 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,580 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,582 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,584 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,585 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,585 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,586 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,586 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,587 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:29:17,587 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-460487761] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,587 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#136548706] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,587 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1300380223 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,587 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1300380223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,587 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-2087594756] 05:29:17,588 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-2087594756] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,591 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,592 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,594 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:29:17,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,595 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,596 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@65bb04bd 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,597 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,597 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,597 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#723634912 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,597 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-532533639] 05:29:17,597 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,597 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-684963180 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,597 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-2108076186] 05:29:17,597 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,598 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-1683627613 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,598 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-1507282861] 05:29:17,598 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,602 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-93014749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,603 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1063546854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,603 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#723634912] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,603 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-532533639] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,603 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-684963180] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,604 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,604 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:29:17,604 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,604 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,604 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,614 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,617 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,618 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,622 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,624 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:17,634 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6413426905630452541] 05:29:17,635 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:17,648 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,649 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,651 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,655 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:17,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:17,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:17,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:17,663 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:17,666 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-906134913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,673 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:17,673 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,674 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,681 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:17,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,684 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,685 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,685 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:17,686 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@51ebdcf5 05:29:17,686 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,687 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,687 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,687 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,688 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:17,690 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,690 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,690 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,690 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] 05:29:17,690 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,691 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,691 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,691 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1797664249] 05:29:17,691 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,691 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1442734603 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1442734603] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,692 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-945129250] 05:29:17,692 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-945129250] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,692 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,693 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#910736321 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,693 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-53244241] 05:29:17,693 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,694 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:29:17,694 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,694 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,694 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:29:17,694 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,694 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1797664249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#910736321] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-53244241] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-609580968] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:17,695 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,695 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,698 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,698 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,698 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,698 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,698 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,698 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 05:29:17,698 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,700 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,702 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,702 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1f40e26f 05:29:17,702 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 05:29:17,703 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:17,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:17,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,704 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1336007989] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,704 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1215352534 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1215352534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:17,704 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#231800078] 05:29:17,705 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,705 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,705 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,705 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,705 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,705 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1336007989] 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#303542400] 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,706 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,706 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,706 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 05:29:17,707 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 05:29:17,707 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#673094076 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,707 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1942890802] 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:29:17,708 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 05:29:17,708 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 05:29:17,708 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 05:29:17,708 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 05:29:17,842 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,843 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 05:29:17,843 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 05:29:17,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:17,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 05:29:17,852 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,853 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 05:29:17,853 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,863 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 05:29:17,863 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 05:29:17,932 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:17,932 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 05:29:17,932 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 05:29:17,933 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 05:29:17,933 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 05:29:18,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,243 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 05:29:18,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,253 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,253 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 05:29:18,253 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,253 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,253 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 05:29:18,333 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,333 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,333 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 05:29:18,484 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:18,484 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,485 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,485 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,486 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,490 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,565 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,566 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,592 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,592 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,592 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 05:29:18,592 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,594 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,594 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:18,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 05:29:18,596 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,596 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 05:29:18,597 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,597 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,598 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,598 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,598 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,598 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,598 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,598 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,599 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,599 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,653 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,653 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,653 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 05:29:18,653 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 05:29:18,653 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,653 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:18,656 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 05:29:18,656 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:18,656 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,656 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:18,656 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:29:18,656 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:18,657 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 05:29:18,657 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 05:29:18,657 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 05:29:18,657 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 05:29:18,657 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 05:29:18,658 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,659 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:29:18,659 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,660 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,660 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,660 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,660 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,660 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,660 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 05:29:18,660 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,660 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,661 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,661 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,661 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,661 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 05:29:18,661 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,661 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,661 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,661 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,661 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,662 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,662 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,662 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,662 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,702 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 05:29:18,702 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,702 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 05:29:18,702 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,702 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,704 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,704 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:18,705 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:18,705 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:18,705 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 05:29:18,705 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,706 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,706 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:18,706 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 05:29:18,706 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 05:29:18,706 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 05:29:18,706 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,706 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 05:29:18,706 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 05:29:18,706 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 05:29:18,706 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,706 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 05:29:18,706 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,706 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,706 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,707 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,707 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 05:29:18,707 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 05:29:18,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#149556980] 05:29:18,707 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:18,707 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,707 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 05:29:18,707 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 05:29:18,707 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 05:29:18,707 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] 05:29:18,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] 05:29:18,707 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1776439863] 05:29:18,707 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,708 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:18,708 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-460487761] 05:29:18,708 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-460487761] 05:29:18,708 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 05:29:18,708 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:18,708 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-906134913] 05:29:18,708 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:18,708 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] 05:29:18,708 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] 05:29:18,711 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 05:29:18,711 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 05:29:18,712 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 05:29:18,712 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:18,713 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,713 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,713 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,713 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards)] 05:29:18,713 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:18,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:29:18,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,714 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards)] 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards)] 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards)] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=22ea3a41-c216-4117-a337-a2ce8a8530cb}, logIndex: 0, isSendImmediate: true 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=2ffcec61-3168-459f-880e-05cb1ea06758}, logIndex: 0, isSendImmediate: true 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,717 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 05:29:18,718 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,717 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a563fcce-cf29-40ed-80c9-5a733c3eb44a}, logIndex: 0, isSendImmediate: true 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 05:29:18,718 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:18,718 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,719 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,719 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,719 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,717 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a11b8710-855f-4fce-b813-774360399899}, logIndex: 0, isSendImmediate: true 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,719 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,722 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,723 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,723 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,723 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 05:29:18,723 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:29:18,723 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,724 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,724 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,724 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,723 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 05:29:18,724 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,724 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 05:29:18,723 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 05:29:18,723 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,724 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,725 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,725 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:29:18,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,726 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,726 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 05:29:18,726 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,726 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:18,726 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 05:29:18,726 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 05:29:18,726 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,726 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,726 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 05:29:18,726 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:29:18,726 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 05:29:18,727 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,727 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,727 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,727 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 05:29:18,727 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:18,728 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 05:29:18,728 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 05:29:18,728 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 05:29:18,728 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#149556980] 05:29:18,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:18,728 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] 05:29:18,728 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#135420377] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#149556980] 05:29:18,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:29:18,729 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1808326840] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-1808326840] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-222806884] 05:29:18,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:18,729 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-619863210] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-619863210] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-222806884] 05:29:18,729 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:29:18,729 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-215822307] 05:29:18,729 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-215822307] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1776439863] 05:29:18,730 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:18,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-460487761] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-460487761] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1776439863] 05:29:18,730 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:29:18,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-93014749] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-93014749] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#136548706] 05:29:18,730 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:29:18,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1063546854] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1063546854] 05:29:18,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#136548706] 05:29:18,731 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:29:18,731 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1076776815] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1076776815] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-906134913] 05:29:18,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:18,731 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#1372275715] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-906134913] 05:29:18,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:29:18,731 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1797664249] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1797664249] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-609580968] 05:29:18,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:18,731 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1336007989] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1336007989] 05:29:18,731 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-609580968] 05:29:18,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:29:18,732 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#303542400] 05:29:18,732 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#303542400] 05:29:18,732 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,732 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:29:18,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,732 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,733 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,733 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,733 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,733 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:18,733 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:18,734 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,734 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,735 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:18,735 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:18,736 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:18,736 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:18,739 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:18,739 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:18,740 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:18,740 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:18,742 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:18,759 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,760 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,760 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:29:18,760 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,760 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,760 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:18,760 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,761 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:18,761 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,761 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:18,761 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:18,762 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:18,762 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:18,762 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,762 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:18,762 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:18,763 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:18,764 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:18,764 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:19,503 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:19,753 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-6413426905630452541] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:19,779 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:19,779 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:19,781 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:19,783 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5810478384434101979] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [10/16/2024 17:29:19.783] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:19,809 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:19,810 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:19,810 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:19,810 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:29:19,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:19,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 05:29:19,810 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:19,810 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 05:29:19,811 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 05:29:19,811 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 05:29:19,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25300317 05:29:19,811 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3aeafb60 05:29:19,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25300317 05:29:19,811 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3aeafb60 05:29:19,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 05:29:19,811 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 05:29:19,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 05:29:19,811 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 05:29:19,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:19,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:19,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:19,812 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:20,192 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 05:29:20,193 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 382.0 ms 05:29:20,193 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 05:29:20,193 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 05:29:20,193 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 05:29:20,194 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 05:29:20,194 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:20,253 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 05:29:20,253 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 442.5 ms 05:29:20,253 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 05:29:20,253 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 05:29:20,253 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 05:29:20,254 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 05:29:20,254 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:20,254 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:20,254 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:20,255 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 05:29:20,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:20,256 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:20,256 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 05:29:20,256 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f0339ed 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@461590d9 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@461590d9 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f0339ed 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 05:29:20,256 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 05:29:20,256 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 05:29:20,257 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 05:29:20,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:20,257 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:20,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:20,257 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:20,572 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 05:29:20,572 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 316.3 ms 05:29:20,573 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 05:29:20,573 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 05:29:20,573 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 05:29:20,573 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 05:29:20,573 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:20,623 PM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 05:29:20,623 PM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 367.0 ms 05:29:20,623 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 05:29:20,623 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 05:29:20,623 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 05:29:20,624 PM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 05:29:20,624 PM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Aborting 0 pending queued transactions 05:29:20,624 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:20,624 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:20,627 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:20,627 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:20,633 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:20.636] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:20,662 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:29:20,672 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5002825973167242205] 05:29:20,672 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:20,675 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:20,675 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:20,677 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:20,677 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:20,690 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:20,690 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:20,690 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:20,690 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,693 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,701 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 05:29:20,707 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:20,709 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,710 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:20,710 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 05:29:20,711 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,711 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 05:29:20,714 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 05:29:20,715 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@177656e 05:29:20,715 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 05:29:20,716 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 05:29:20,717 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,717 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 05:29:20,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 05:29:20,717 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1104682419] 05:29:20,718 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-default-config_testGetPrefixShardRole, persistent : true 05:29:20,718 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,719 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-default-config_testGetPrefixShardRole with journal batch size 1 05:29:20,719 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1104682419] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,719 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-8218693] 05:29:20,719 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#1453681951 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 05:29:20,719 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#1453681951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:20,720 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 05:29:20,720 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 05:29:20,720 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 05:29:20,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-config_testGetPrefixShardRole 05:29:20,721 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 05:29:20,721 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 05:29:20,721 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:20,721 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,721 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 05:29:20,721 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Received ElectionTimeout 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 05:29:20,722 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 05:29:20,722 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 05:29:20,722 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole 05:29:20,723 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config_testGetPrefixShardRole, leaderId=member-1-shard-default-config_testGetPrefixShardRole, leaderPayloadVersion=13] 05:29:20,723 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 05:29:20,723 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 05:29:20,723 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#-8218693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,723 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1140855447] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,724 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:20,724 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,724 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,724 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,725 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 05:29:20,726 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 05:29:20,726 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@65ab70c7 05:29:20,726 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,726 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,727 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-default-oper_testGetPrefixShardRole, persistent : true 05:29:20,727 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-default-oper_testGetPrefixShardRole with journal batch size 1 05:29:20,727 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-1156435748] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,727 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#75303305] 05:29:20,727 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#75303305] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,727 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#563422195 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,727 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#563422195] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-1156435748] 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:20,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 05:29:20,728 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 05:29:20,728 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,728 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 05:29:20,729 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Received ElectionTimeout 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 05:29:20,729 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 05:29:20,729 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener akka://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-oper_testGetPrefixShardRole, leaderId=member-1-shard-default-oper_testGetPrefixShardRole, leaderPayloadVersion=13] 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 05:29:20,729 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 05:29:20,776 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] 05:29:20,776 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 05:29:20,777 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#1104682419] 05:29:20,777 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 05:29:20,777 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 05:29:20,777 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onShutDown 05:29:20,777 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@79681587 05:29:20,777 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@79681587 05:29:20,778 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-default-config_testGetPrefixShardRole 05:29:20,778 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Aborting 0 pending queued transactions 05:29:20,778 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 05:29:20,779 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:20,779 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:20,779 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 05:29:20,779 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onShutDown 05:29:20,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@71c08a61 05:29:20,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@71c08a61 05:29:20,780 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,780 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-default-oper_testGetPrefixShardRole 05:29:20,780 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Aborting 0 pending queued transactions 05:29:20,780 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RepointableActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-config_testGetPrefixShardRole#2053512642] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,780 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 05:29:20,781 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:20,781 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:20,782 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:20,782 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:20,784 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:20,800 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:20,807 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2468069618224929414] 05:29:20,808 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:20,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:20,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:20,817 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:20,817 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:20,818 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,819 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,821 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:20,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:20,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:20,824 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,825 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:20,826 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:20,826 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,827 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,827 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,827 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,827 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:20,828 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:20,828 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@53c853aa 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,829 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:20,829 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,829 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,830 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,831 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,831 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#286078223] 05:29:20,831 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1414973672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,831 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,831 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,831 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-647174164 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,831 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#-2145688750] 05:29:20,832 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,834 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,834 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,834 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#224265895] 05:29:20,834 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,836 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1414973672 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,836 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:20,836 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#286078223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#224265895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-647174164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#-2145688750] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,838 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-777122769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,842 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,845 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,847 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:20,847 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,848 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:20,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,848 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,848 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,849 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,849 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:20,850 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52618b3b 05:29:20,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-215585361] 05:29:20,851 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,852 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,852 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#476788399] 05:29:20,852 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1775678737 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,852 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,852 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1775678737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,853 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#240884243 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,853 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,853 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#577348082] 05:29:20,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:20,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,856 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,857 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:20,857 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,857 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,857 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,857 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,860 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:20,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2583284975108035521] 05:29:20,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:29:20,870 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:20,870 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:29:20,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:20,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:20,883 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,886 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,890 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,894 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1582857837] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,898 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:29:20,903 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:29:20,903 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,903 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,905 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:20,907 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4e1015c1 05:29:20,907 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:20,907 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,908 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,911 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,914 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:29:20,915 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,915 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,915 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1832525388] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,916 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1310416533 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,916 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,916 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1310416533] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,916 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,916 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1320705865] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,916 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#727486978 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,916 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1000085019] 05:29:20,917 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,918 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:20,922 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#727486978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,922 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1000085019] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,923 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-995202272] 05:29:20,923 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-995202272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,923 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,927 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,931 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:29:20,931 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:29:20,931 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,931 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-223788495] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,934 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:29:20,934 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,935 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,935 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:20,935 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,935 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,935 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:29:20,935 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,936 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,936 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,936 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,936 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,936 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,936 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,936 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,936 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,936 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,936 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,936 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,936 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:20,937 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:20,937 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4a409d5b 05:29:20,937 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:20,937 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,938 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,939 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,939 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1801435499] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,940 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2039817823 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-2118129329] 05:29:20,940 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2039817823] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,940 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,940 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,940 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1801435499] 05:29:20,940 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,940 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,941 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,941 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:20,941 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-2095808074 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,942 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#1743813321] 05:29:20,942 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1760855730] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,942 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:20,942 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:29:20,942 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:20,942 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:20,942 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,942 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,943 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:20,946 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:20,954 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2208659299614907688] 05:29:20,955 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:20,957 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:20,957 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:20,958 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:20,958 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:20,972 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,974 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:20,980 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:20,989 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:20,993 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1915682829] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:20,996 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:21,0 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:21,0 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,0 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:21,1 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,2 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:21,2 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,2 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,2 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5bf1332e 05:29:21,2 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:21,3 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:21,3 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,3 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:21,4 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:21,4 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:21,4 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,4 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,4 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-779151904] 05:29:21,4 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:21,4 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,5 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-303985820] 05:29:21,5 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#524410728 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,5 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#206003088] 05:29:21,5 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:21,5 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:21,5 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#13586185 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,5 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-615264538] 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-779151904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#524410728] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#206003088] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,6 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-303985820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#13586185] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-615264538] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,7 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#312068546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:29:21,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:21,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:21,9 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:29:21,9 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,10 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,10 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:21,11 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,11 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:21,11 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:21,33 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,33 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 05:29:21,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6d55092e 05:29:21,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:21,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,35 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:21,35 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#234172976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#234172976] 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1189468381 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#766303957] 05:29:21,36 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1189468381] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:21,36 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,36 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1012178883] 05:29:21,37 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 05:29:21,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:21,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:21,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:21,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 05:29:21,37 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:21,38 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 05:29:21,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:21,38 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,38 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-1700402183] 05:29:21,38 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:21,38 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,38 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#979314924 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,38 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:21,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:29:21,39 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,830 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,830 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,836 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] 05:29:21,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:21,836 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,836 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,838 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] 05:29:21,838 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:21,839 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,839 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,839 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,839 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,840 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 05:29:21,840 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:29:21,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:29:21,841 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:29:21,841 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:21,841 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:29:21,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:29:21,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:21,841 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:29:21,842 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,842 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 05:29:21,842 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,843 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:21,843 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:21,843 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:21,843 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:21,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,843 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,843 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,843 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 05:29:21,843 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,843 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,843 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,843 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 05:29:21,843 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 05:29:21,843 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 05:29:21,843 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 05:29:21,843 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 05:29:21,843 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 05:29:21,843 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 05:29:21,843 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 05:29:21,843 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,844 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,844 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,844 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,844 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,844 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,844 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=485926c3-d1b5-4719-9ae3-955cb628ea9c}, logIndex: 1, isSendImmediate: true 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,847 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,847 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=bd1a0cd7-1051-418c-9123-47ca87189710}, logIndex: 1, isSendImmediate: true 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:29:21,848 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader elected 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,848 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,849 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=2f8ab76e-ea03-49a3-906e-a7d8332aa6d7}, logIndex: 1, isSendImmediate: true 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,849 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,849 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 05:29:21,849 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,850 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 05:29:21,850 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:29:21,850 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,850 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 05:29:21,850 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 05:29:21,850 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,850 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,850 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,851 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 05:29:21,851 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,851 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,851 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 05:29:21,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,852 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,852 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,852 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,852 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,852 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader elected 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=f86ccb9f-a381-418c-aa23-541ac0d2f856}, logIndex: 1, isSendImmediate: true 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,853 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:29:21,853 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,853 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 05:29:21,853 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,854 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,854 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,854 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,854 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,855 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 05:29:21,855 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,855 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,855 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,855 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,856 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:29:21,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,856 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#224265895] 05:29:21,856 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#224265895] 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,856 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,857 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 2, lastIndex: 2 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 2, lastIndex: 2 05:29:21,857 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,857 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,857 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,857 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,858 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,858 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,859 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 05:29:21,859 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 05:29:21,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,857 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 05:29:21,857 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-777122769] 05:29:21,859 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 05:29:21,859 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,859 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,859 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,859 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 2, lastIndex: 2 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,860 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,860 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,860 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 05:29:21,860 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 05:29:21,860 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-777122769] 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,860 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,860 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-215585361] 05:29:21,860 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-215585361] 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1582857837] 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,861 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,861 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:21,861 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1832525388] 05:29:21,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1832525388] 05:29:21,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1582857837] 05:29:21,861 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:29:21,861 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1320705865] 05:29:21,861 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1320705865] 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 2, lastIndex: 2 05:29:21,862 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 05:29:21,862 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,862 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,863 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,863 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,863 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,863 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,863 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,864 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 05:29:21,864 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:21,864 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 05:29:21,864 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 05:29:21,864 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 05:29:21,864 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:21,864 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] is present: existingEntryTerm: 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,864 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:21,865 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,865 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 05:29:21,865 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 05:29:21,865 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:21,901 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,901 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:21,901 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,902 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,903 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1320705865] 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-223788495] 05:29:21,912 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:29:21,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1801435499] 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1801435499] 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-223788495] 05:29:21,912 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:29:21,912 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1760855730] 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1760855730] 05:29:21,912 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1915682829] 05:29:21,913 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:21,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-779151904] 05:29:21,913 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-779151904] 05:29:21,913 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1915682829] 05:29:21,913 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:29:21,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-303985820] 05:29:21,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-303985820] 05:29:21,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#312068546] 05:29:21,914 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:21,914 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#234172976] 05:29:21,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#234172976] 05:29:21,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#312068546] 05:29:21,914 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:29:21,914 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1012178883] 05:29:21,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1012178883] 05:29:21,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-887516267] 05:29:21,915 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:21,915 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,915 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#277373811] 05:29:21,916 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-777122769] 05:29:21,916 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:21,916 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] 05:29:21,916 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-482125836] 05:29:21,916 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,917 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:29:21,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,918 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,918 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,918 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,918 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:21,918 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:21,919 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,919 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,919 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,920 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,920 PM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,920 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,920 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,920 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:29:21,920 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:21,920 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:21,921 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:21,921 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:21,925 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:21,925 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:21,925 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:21,925 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:21,926 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:21,942 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,943 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,943 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,944 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:21,944 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:21,944 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:21,945 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,945 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,945 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:29:21,945 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,945 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:21,946 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,946 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:21,946 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,946 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:21,946 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:21,946 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:21,947 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:21,947 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:22,848 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:22,937 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2208659299614907688] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:22,968 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:22,968 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:22,968 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2583284975108035521] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:22,970 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:22.971] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:22,976 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:22,977 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:22,977 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:22,977 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 05:29:22,977 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:22,977 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 05:29:22,977 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 05:29:22,977 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e16f40e 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e16f40e 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 05:29:22,977 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:22,978 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1166a087 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1166a087 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:22,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:23,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 05:29:23,448 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 470.7 ms 05:29:23,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 05:29:23,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 05:29:23,448 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 05:29:23,449 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,449 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:23,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 05:29:23,458 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 480.7 ms 05:29:23,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 05:29:23,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 05:29:23,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 05:29:23,458 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,459 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:23,459 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:23,459 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:23,459 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@549d34a9 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75746dd4 05:29:23,460 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@549d34a9 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@75746dd4 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 05:29:23,460 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:23,460 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:23,460 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:23,777 PM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 05:29:23,777 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 05:29:23,777 PM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.4 ms 05:29:23,777 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 317.5 ms 05:29:23,778 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 05:29:23,778 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 05:29:23,778 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 05:29:23,778 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 05:29:23,778 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 05:29:23,778 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 05:29:23,778 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,778 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 05:29:23,778 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:23,778 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Aborting 0 pending queued transactions 05:29:23,778 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:23,779 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:23,780 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:23,780 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:23,783 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:23.785] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:23,814 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:29:23,825 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2076344624248359681] 05:29:23,825 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:23,836 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:23,836 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:23,836 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:23,836 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:23,841 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,842 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:23,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:23,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:23,844 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,847 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:23,849 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1294908613] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,880 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:23,881 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@26cb078c 05:29:23,881 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:23,882 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:23,882 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,882 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,883 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1899847198] 05:29:23,884 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,884 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,884 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,884 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,885 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,886 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,886 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1899847198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,887 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#996191744 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,887 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#996191744] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,887 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,888 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-1526812055] 05:29:23,888 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-1526812055] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,888 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2100957990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,889 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:23,889 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-1159756981] 05:29:23,889 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-1159756981] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,890 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:23,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,891 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,891 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1268927362 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,893 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:23,897 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1268927362] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,898 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:23,898 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,898 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,898 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,898 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,899 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,899 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:23,899 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,899 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@322a6166 05:29:23,899 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,899 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-75827313] 05:29:23,900 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,900 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,901 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,901 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1387608014 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,902 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#348861726] 05:29:23,903 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-75827313] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,903 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1387608014] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,903 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,903 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,903 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-682343742] 05:29:23,903 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,904 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-510443399] 05:29:23,904 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1944471062 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,904 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,905 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:23,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,905 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,906 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:23,906 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,906 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,906 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:23,907 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,907 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,908 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,908 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,908 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,908 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:23,908 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,908 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,908 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,915 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 05:29:23,925 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8567278422905591731] 05:29:23,926 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:29:23,928 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:23,928 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:29:23,934 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:23,934 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:23,938 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,940 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,941 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,947 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-276404687] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,948 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:29:23,953 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:29:23,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,953 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,957 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:23,958 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2bc42ca4 05:29:23,958 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:23,959 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:23,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,960 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:23,962 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,962 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,963 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,963 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1215320901] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,963 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-2117164219] 05:29:23,964 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#157598928] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,966 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,967 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,967 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#46739996 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,967 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-321232224] 05:29:23,968 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#157598928 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,968 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:29:23,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,968 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,968 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,968 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,969 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:29:23,969 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,969 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,970 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:29:23,971 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,972 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,972 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:29:23,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,972 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:23,972 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,972 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,972 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,975 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-2117164219] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,975 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#447424468] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,975 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#704929986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,975 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#46739996] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,975 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-321232224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,982 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:23,983 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@51850513 05:29:23,983 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,984 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1168393721] 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,991 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#219684137] 05:29:23,992 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:23,992 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,992 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:23,994 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1444727396 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,994 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#1034049011] 05:29:23,995 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#606330840] 05:29:23,995 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1168393721] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,995 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#556094367 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,995 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#219684137] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:23,995 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,995 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:23,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:29:23,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:23,996 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:23,996 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:23,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:29:23,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:23,996 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,996 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:23,996 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,996 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:23,996 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:24,1 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:24,9 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6954489203208587359] 05:29:24,9 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:24,12 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:24,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:24,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:24,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:24,29 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:24,35 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:24,41 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,48 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:24,49 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:24,50 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1872703410] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,54 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,57 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,61 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:24,61 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@28b0e9b2 05:29:24,61 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:24,62 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:24,62 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,62 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:24,63 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:24,65 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,67 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,67 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,67 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1141580262] 05:29:24,67 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,68 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,68 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,68 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-170575707] 05:29:24,68 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:24,69 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:29:24,69 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1318749519] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1141580262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-170575707] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1872218632] 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-458738014 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-458738014] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1349815366 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,69 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#551984743] 05:29:24,69 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:24,70 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:24,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:24,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:24,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:24,71 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1872218632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,71 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1349815366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,71 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#551984743] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 05:29:24,71 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:24,72 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:24,72 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:24,72 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,73 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,73 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:24,73 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,73 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,73 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,73 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,73 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:24,74 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:24,74 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:24,72 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:24,74 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,74 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:24,74 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:24,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:29:24,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:24,74 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:24,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,75 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32b9c64 05:29:24,75 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:24,75 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 05:29:24,76 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#127806341] 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,76 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-602955156] 05:29:24,77 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:24,77 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#127806341] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,77 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1225958256 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,77 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1492668614] 05:29:24,77 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-602955156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:24,77 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,78 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,78 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:24,78 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:24,78 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1901417522 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:24,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:24,79 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,79 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:24,79 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,79 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:24,79 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-1414592776] 05:29:24,79 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 05:29:24,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:29:24,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,80 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 05:29:24,80 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,80 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,80 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 05:29:24,226 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:24,226 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,227 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,227 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 05:29:24,228 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,257 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 05:29:24,257 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 05:29:24,306 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:24,306 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,307 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 05:29:24,307 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 05:29:24,326 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 05:29:24,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 05:29:24,327 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 05:29:24,327 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 05:29:24,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,607 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 05:29:24,637 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,637 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,637 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 05:29:24,646 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,646 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,646 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 05:29:24,677 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,677 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,677 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 05:29:24,847 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:24,847 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:29:24,848 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,848 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,848 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,849 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,854 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,854 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,964 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,966 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,966 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,966 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 05:29:24,966 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,968 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,968 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,969 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 05:29:24,969 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,970 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 05:29:24,970 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,970 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,970 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:24,970 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,971 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,971 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,971 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,971 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:24,971 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:24,971 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,972 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,972 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,972 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,973 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:24,973 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 05:29:24,973 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 05:29:24,973 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:24,973 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 4 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,974 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:24,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:24,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:24,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,987 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 05:29:24,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,987 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,988 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:24,988 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 05:29:24,989 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:29:24,989 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,990 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,990 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,990 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 05:29:24,990 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,990 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,991 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:24,991 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:24,991 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,990 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:24,991 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:24,991 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:24,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:24,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,991 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,993 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,994 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,994 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,994 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:24,995 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 05:29:24,995 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:24,995 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 4 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,996 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:24,996 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,26 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:25,26 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,26 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 05:29:25,26 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,27 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,27 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:25,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:29:25,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:29:25,29 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:25,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:25,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:25,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:25,30 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:25,30 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 05:29:25,30 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:29:25,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,30 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 05:29:25,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,31 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,31 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,31 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:25,32 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:25,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,32 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:25,33 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,33 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,34 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,34 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:25,34 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:25,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,35 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:25,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,36 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,37 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:25,37 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 05:29:25,37 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,37 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,37 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 05:29:25,37 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] is present: existingEntryTerm: 4 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,47 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 05:29:25,47 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,47 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 05:29:25,47 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,47 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 05:29:25,48 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 05:29:25,49 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,49 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,49 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,50 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 05:29:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 05:29:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:25,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:25,50 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,50 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,51 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,51 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,51 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,51 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,51 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,51 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,52 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:25,52 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,52 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]]] 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,53 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,53 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023] 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,53 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023]] - lastApplied: 1, lastIndex: 1 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 05:29:25,54 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,54 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@2915b023 05:29:25,54 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 05:29:25,54 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 05:29:25,55 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 05:29:25,55 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 05:29:25,55 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 05:29:25,55 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1294908613] 05:29:25,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:25,56 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,56 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,56 PM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 05:29:25,56 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:29:25,56 PM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,56 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 05:29:25,56 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:29:25,56 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 05:29:25,57 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:25,57 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:25,57 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:29:25,57 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:29:25,57 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]], state: Idle 05:29:25,57 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:25,57 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,57 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,57 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 05:29:25,57 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 05:29:25,57 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,57 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,57 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,57 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,57 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=1dfb1000-d837-48f2-9143-a5724db6d9da}, logIndex: 2, isSendImmediate: true 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=6c41785a-f79b-41be-b48c-6891e5ea1984}, logIndex: 2, isSendImmediate: true 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=451113ae-84a2-48a4-afe7-e1846f1c1c19}, logIndex: 2, isSendImmediate: true 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=28b4b79d-5a30-4f3d-bc71-23c542f56992}, logIndex: 2, isSendImmediate: true 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,58 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 05:29:25,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 05:29:25,58 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 05:29:25,58 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,59 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,59 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,59 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,59 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 05:29:25,59 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 05:29:25,59 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 05:29:25,59 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,59 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,59 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,59 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,58 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,60 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,60 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 05:29:25,60 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 05:29:25,60 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,60 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1294908613] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,60 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:25,61 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1294908613] 05:29:25,61 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 05:29:25,61 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1899847198] 05:29:25,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1899847198] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2100957990] 05:29:25,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 05:29:25,61 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-75827313] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-75827313] 05:29:25,61 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-2100957990] 05:29:25,61 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 05:29:25,61 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-682343742] 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-682343742] 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-276404687] 05:29:25,62 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:25,62 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1215320901] 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1215320901] 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-276404687] 05:29:25,62 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 05:29:25,62 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#704929986] 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#704929986] 05:29:25,62 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,62 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,62 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,62 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,62 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,62 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,62 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#447424468] 05:29:25,63 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 05:29:25,63 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1168393721] 05:29:25,63 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1168393721] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,63 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,64 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,64 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:25,64 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#447424468] 05:29:25,64 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 05:29:25,64 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 05:29:25,64 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#219684137] 05:29:25,64 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,64 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,64 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#219684137] 05:29:25,65 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,65 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 05:29:25,65 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,65 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1872703410] 05:29:25,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:25,65 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1141580262] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1141580262] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1872703410] 05:29:25,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 05:29:25,65 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-170575707] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-170575707] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1318749519] 05:29:25,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 05:29:25,65 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#127806341] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#127806341] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1318749519] 05:29:25,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 05:29:25,65 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-602955156] 05:29:25,65 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-602955156] 05:29:25,66 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1294908613] 05:29:25,66 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:25,66 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,66 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1776939680] 05:29:25,66 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:25,66 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,66 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,66 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:29:25,66 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,66 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,67 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,67 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,67 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,67 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,67 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:25,67 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:25,67 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:25,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,68 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:29:25,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,68 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,68 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,68 PM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:25,69 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:25,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:25,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:25,73 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:25,73 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:25,73 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:25,73 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:25,74 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:25,97 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:25,97 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,97 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,97 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:29:25,97 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,97 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,98 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,98 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,98 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,98 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,98 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:25,98 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:25,99 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:25,100 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,100 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,100 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,100 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:29:25,100 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:25,100 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,100 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:25,100 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:25,101 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:25,101 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:25,103 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:25,103 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:25,676 PM [cluster-test-shard-dispatcher-34] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 05:29:25,676 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,676 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,677 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 05:29:25,867 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:26,87 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [6954489203208587359] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:26,117 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:26,117 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:26,118 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8567278422905591731] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:26,119 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:26.121] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:26,126 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:26,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,126 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:29:26,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:26,126 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 05:29:26,126 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 05:29:26,126 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60ec3225 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@60ec3225 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25966f0e 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@25966f0e 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,127 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 05:29:26,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 05:29:26,567 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 440.2 ms 05:29:26,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 05:29:26,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 05:29:26,567 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 05:29:26,568 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,568 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:26,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 05:29:26,636 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 509.7 ms 05:29:26,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 05:29:26,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 05:29:26,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 05:29:26,637 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:26,637 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:26,638 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:26,638 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 05:29:26,638 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,638 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,638 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e729b0b 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e729b0b 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ba72bc8 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ba72bc8 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 05:29:26,639 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 05:29:26,966 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 05:29:26,966 PM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 327.7 ms 05:29:26,966 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 05:29:26,966 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 05:29:26,967 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 05:29:26,967 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:26,967 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 588.1 ms 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 05:29:27,227 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 05:29:27,228 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Aborting 0 pending queued transactions 05:29:27,228 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:27,228 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:27,229 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:27,229 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:27,231 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:27.234] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:27,257 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:27,268 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-466716471271649135] 05:29:27,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:27,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:27,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:27,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:27,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:27,286 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,288 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:27,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:27,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:27,290 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 05:29:27,292 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1717000960] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,294 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:27,295 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:27,295 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,296 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,296 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,300 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:27,301 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20a207d0 05:29:27,301 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 05:29:27,301 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,301 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,301 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,302 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,302 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,303 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,303 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1696401254 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,303 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,303 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1696401254] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,302 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,305 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,308 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#614411912] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,308 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1765336022] 05:29:27,310 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] 05:29:27,310 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,310 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1765336022] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,310 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:27,310 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,311 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#2109348685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,311 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1541230412] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,311 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-100164448] 05:29:27,311 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-100164448] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,311 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,312 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1541230412 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,312 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:27,312 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,313 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,313 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,313 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,313 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,315 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:27,315 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 05:29:27,318 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@175e2e7a 05:29:27,318 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,318 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,319 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,319 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-1839092697] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,320 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-125047318 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,320 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-981814866] 05:29:27,320 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,320 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-125047318] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,320 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,320 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,320 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,320 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,320 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,320 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,321 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,320 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-1839092697] 05:29:27,321 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,321 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,322 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,322 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-2098157745] 05:29:27,322 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,322 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,322 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,322 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,322 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#285743164 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,322 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1873071366] 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 05:29:27,322 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:27,322 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 05:29:27,322 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,323 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,323 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#2109348685] 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 05:29:27,324 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,324 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,324 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,330 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:27,341 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2376387042288767907] 05:29:27,342 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:29:27,351 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:27,352 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:29:27,352 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:27,352 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:27,357 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,358 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,361 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 05:29:27,369 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:29:27,370 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1202780666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,379 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:29:27,379 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,379 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:27,382 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:27,384 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 05:29:27,384 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,389 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,391 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,398 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@734d8ee9 05:29:27,398 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,398 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,399 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1549528286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,400 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:29:27,400 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:27,400 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,400 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1368805905 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,401 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-544678349] 05:29:27,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1368805905] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,401 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 05:29:27,401 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-544678349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,401 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,403 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:29:27,403 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,403 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,403 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@645a82b0 05:29:27,404 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,404 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,404 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:27,406 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,406 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-777693065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,406 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1745048671 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:27,406 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1745048671] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,406 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#1866166588] 05:29:27,406 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#1866166588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,407 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,407 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,407 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-1067735233] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,407 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1503950153 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,407 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1869436006] 05:29:27,407 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1503950153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,408 PM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,408 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,409 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,409 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,409 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#541336025] 05:29:27,409 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,409 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:29:27,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 05:29:27,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,410 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,410 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:29:27,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 05:29:27,410 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,410 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,410 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,410 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,410 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:29:27,410 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 05:29:27,410 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,410 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,411 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1015531452 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,411 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,411 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,411 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,411 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,411 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,411 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:29:27,411 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 05:29:27,411 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,411 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,412 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,412 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,414 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,414 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,414 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,414 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,414 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,419 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:27,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:29:27,428 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:27,428 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,428 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,428 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,428 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,429 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,442 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3636933762823189940] 05:29:27,443 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:27,445 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:27,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:27,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:27,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:27,457 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,458 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,460 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 05:29:27,471 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:27,471 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1874213990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,473 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:27,473 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:27,475 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:27,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3ff9af55 05:29:27,476 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 05:29:27,476 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,477 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:27,479 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,481 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:27,482 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,482 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1086015656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,482 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,483 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#410040079 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,483 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-2091412401] 05:29:27,483 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardLeaderReplica, persistent : true 05:29:27,483 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#912566666 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:27,483 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-863036320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,485 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:29:27,488 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,488 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,489 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:29:27,489 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,489 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,489 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:27,489 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 05:29:27,490 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1137294958] 05:29:27,491 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#410040079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#-1154599011] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-2091412401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#912566666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1137294958] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,491 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:27,492 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 05:29:27,492 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:27,493 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a228c4f 05:29:27,493 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:27,493 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,494 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,494 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-7223693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,494 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1705634271 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,494 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,494 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 05:29:27,494 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1705634271] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:27,495 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#417689458] 05:29:27,495 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#453352633 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,495 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1428127841] 05:29:27,495 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 05:29:27,499 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,499 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,499 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,499 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,499 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,503 PM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:29:27,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 05:29:27,503 PM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,503 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:29:27,503 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:27,503 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:27,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,503 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,503 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 05:29:27,504 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:27,507 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,507 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,512 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 05:29:27,513 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 05:29:27,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:27,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 05:29:27,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 05:29:27,645 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:27,645 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 05:29:27,645 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:27,645 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,645 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,645 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:27,646 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 05:29:27,663 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:27,663 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:27,664 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,714 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:27,714 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:27,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 05:29:27,715 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 05:29:27,715 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:27,715 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 05:29:28,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,34 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,34 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 05:29:28,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,44 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 05:29:28,103 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,104 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,104 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 05:29:28,134 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,134 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,134 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 05:29:28,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:28,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,286 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,293 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,294 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,294 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,294 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,385 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,385 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,385 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,385 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to akka://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,386 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,403 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,403 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,403 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,403 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,403 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 05:29:28,403 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 05:29:28,403 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,403 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,404 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,404 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,405 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,405 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,405 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,405 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,406 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,406 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,406 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,406 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,406 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,406 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,406 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,408 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,408 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,408 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,408 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,409 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,409 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,409 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,409 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 05:29:28,410 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,410 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,410 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,410 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 05:29:28,410 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,411 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,411 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,411 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,411 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,411 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 05:29:28,411 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:28,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,411 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,411 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 05:29:28,411 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,411 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,465 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:29:28,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,483 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 05:29:28,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,485 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,485 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,486 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,486 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,487 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,487 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,487 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,487 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 05:29:28,487 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1717000960] 05:29:28,487 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:28,487 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] 05:29:28,487 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] 05:29:28,487 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,488 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,488 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1202780666] 05:29:28,488 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 05:29:28,488 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:28,488 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,488 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,488 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,488 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1874213990] 05:29:28,488 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,488 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,488 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 05:29:28,488 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:28,489 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1086015656] 05:29:28,489 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1086015656] 05:29:28,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,489 PM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-1, datastoreType Config 05:29:28,489 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 05:29:28,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 05:29:28,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7df5b03d] 05:29:28,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onRemoveServer: RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'}, state: Idle 05:29:28,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]] 05:29:28,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bbcc044e-f14b-4e70-b8d5-b5b70ddc8838}, logIndex: 0, isSendImmediate: true 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: Returning OK for operation RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'} 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 05:29:28,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,491 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 05:29:28,491 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,491 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 05:29:28,492 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:28,492 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 05:29:28,492 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 05:29:28,492 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:28,493 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,493 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testRemoveShardLeaderReplica, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:28,493 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1202780666] 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:28,493 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Setting last applied to 0 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 05:29:28,493 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 05:29:28,493 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] with 10000 ms timeout 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 05:29:28,494 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 05:29:28,494 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 05:29:28,495 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,494 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 05:29:28,495 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Successfully deleted prior snapshots 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 05:29:28,495 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 05:29:28,495 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@733251f3 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@733251f3 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,495 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 05:29:28,498 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 05:29:28,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 05:29:28,499 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] is present: existingEntryTerm: 3 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 1 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,499 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:28,500 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 05:29:28,500 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 05:29:28,500 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 3 - updating term 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=null 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,501 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,502 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,502 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,502 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,502 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,502 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,502 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-3-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,503 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 05:29:28,503 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,503 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,503 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 05:29:28,503 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,503 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 05:29:28,503 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,504 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 05:29:28,504 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,504 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,504 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 05:29:28,504 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,504 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,504 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,505 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,505 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,505 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,505 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,505 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:28,505 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,506 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,506 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 05:29:28,507 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,507 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 05:29:28,507 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,507 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 05:29:28,507 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,507 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,507 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,508 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 05:29:28,508 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-1-shard-people-oper_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,508 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 05:29:28,508 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:28,508 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 05:29:28,548 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,548 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1202780666] 05:29:28,549 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:28,549 PM [cluster-test-akka.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,549 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#386929790] 05:29:28,549 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1874213990] 05:29:28,549 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:28,549 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1086015656] 05:29:28,549 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-1086015656] 05:29:28,550 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1717000960] 05:29:28,550 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:28,550 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 05:29:28,550 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:28,550 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@112bd975 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-3-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@112bd975 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,551 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onShutDown 05:29:28,552 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 05:29:28,552 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardLeaderReplica 05:29:28,552 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:28,553 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,553 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 05:29:28,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 05:29:28,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 05:29:28,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 05:29:28,554 PM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 5 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,555 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 5 05:29:28,556 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:28,556 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:28,556 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 05:29:28,556 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 05:29:28,556 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cluster state: [] 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer in progress - processing RequestVote 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 3 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:28,557 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:28,558 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 05:29:28,558 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 05:29:28,558 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 05:29:28,558 PM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:28,559 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 05:29:28,559 PM [cluster-test-akka.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 05:29:28,559 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,559 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 05:29:28,559 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,559 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: getLeaderAddress leaderId = member-2-shard-cars-config_testRemoveShardLeaderReplica peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,559 PM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader changed to member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,559 PM [cluster-test-shard-dispatcher-32] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to member-2-shard-cars-config_testRemoveShardLeaderReplica in 8.549 ms 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:28,560 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:28,560 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:28,561 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:28,561 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 05:29:28,561 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,562 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:29:28,562 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 05:29:28,562 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 05:29:28,562 PM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 05:29:28,562 PM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:28,562 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 05:29:28,562 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:28,562 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:28,562 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:28,563 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:28,563 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:28,566 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:28,567 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:28,567 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:28,567 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:28,569 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:28,586 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:28,586 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,586 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,586 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50359834 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50359834 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:28,587 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onShutDown 05:29:28,588 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 05:29:28,588 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardLeaderReplica 05:29:28,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:28,726 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 05:29:28,729 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:28,805 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 5 05:29:28,806 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "AppendEntriesReply [term=5, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 05:29:28,806 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 05:29:28,806 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 05:29:28,807 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 05:29:28,807 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 05:29:28,807 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 05:29:28,807 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 05:29:28,807 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:28,807 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 05:29:28,825 PM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 05:29:28,826 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:29,47 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:29,147 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:29,304 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:29,365 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:29:29,465 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:29,579 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3636933762823189940] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:29,583 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-3636933762823189940] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:29,686 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:29,786 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:30,7 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:29:30,106 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:30,327 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:30,425 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:30,513 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 05:29:30,513 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.018 s 05:29:30,513 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 05:29:30,514 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 05:29:30,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:30,514 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[akka://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1499268215] 05:29:30,647 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:30,746 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:29:30,967 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:31,66 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:31,286 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:31,386 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:31,606 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:31,706 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:31,927 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:32,26 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:32,246 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:32,347 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:32,566 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:32,666 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:32,887 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:32,888 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:32,986 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:33,207 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:33,306 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:33,410 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:33,524 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,524 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:33,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 05:29:33,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 05:29:33,525 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,525 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:33,527 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 05:29:33,527 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 05:29:33,624 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,624 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 05:29:33,625 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,625 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,625 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:29:33,625 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:33,625 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 05:29:33,625 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:33,627 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:33,627 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 05:29:33,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 05:29:33,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,627 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:33,628 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 05:29:33,628 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:33,628 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 05:29:33,628 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 05:29:33,628 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 05:29:33,846 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:29:33,849 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:33,946 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 05:29:34,170 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 324, lastApplied : -1, commitIndex : -1 05:29:34,170 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:34,266 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:34,405 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 05:29:34,405 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:34,405 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:34,405 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:34,409 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:34,409 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:34,430 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:34,486 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 05:29:34,489 PM [cluster-test-akka.actor.default-dispatcher-33] [WARN] Cluster - akkaUnreachable Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 05:29:34,489 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:34,586 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:34,806 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:34,810 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:34,906 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:35,128 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:29:35,129 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:35,225 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:35,448 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:35,450 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:35,454 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:35,546 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:35,766 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:35,866 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:36,87 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:36,186 PM [cluster-test-shard-dispatcher-14] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:36,235 PM [cluster-test-akka.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:36,407 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:36,505 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 05:29:36,731 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:36,731 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 324, lastApplied : -1, commitIndex : -1 05:29:36,826 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:37,46 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 05:29:37,50 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:37,146 PM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 05:29:37,367 PM [cluster-test-shard-dispatcher-31] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 05:29:37,467 PM [cluster-test-shard-dispatcher-31] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 05:29:37,488 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 05:29:37,685 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:29:37,785 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 05:29:37,829 PM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 05:29:37,829 PM [cluster-test-shard-dispatcher-35] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.242 s 05:29:37,829 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 05:29:37,829 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer failed - sending PoisonPill 05:29:37,829 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer expired 05:29:37,830 PM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 05:29:37,830 PM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:37,830 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:37,830 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:37,830 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:37,831 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:37,831 PM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:37,831 PM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:29:37,831 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 05:29:37,831 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 05:29:37,831 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 05:29:37,831 PM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 05:29:37,831 PM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:37,831 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:37,831 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:37,831 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:37,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:37,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:38,644 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 05:29:38,644 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 05:29:38,744 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 05:29:38,744 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 05:29:38,852 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:38,852 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:38,853 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:38,854 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2376387042288767907] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [10/16/2024 17:29:38.855] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:38,869 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 05:29:38,870 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:38,871 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 05:29:38,871 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onShutDown 05:29:38,871 PM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardLeaderReplica 05:29:38,871 PM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:38,872 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:38,872 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:38,872 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 05:29:38,872 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:38,872 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:38,873 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:38,873 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 05:29:38,873 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 05:29:38,873 PM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 05:29:38,873 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 05:29:38,873 PM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:38,873 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Aborting 0 pending queued transactions 05:29:38,873 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:38,873 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:38,875 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:38,875 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:38,876 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:38.877] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:38,891 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 05:29:38,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8276909700362377003] 05:29:38,901 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:38,907 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:38,907 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:38,907 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:38,907 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:38,911 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,912 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,914 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,915 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:38,915 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:38,915 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:38,918 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-395863976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,918 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:38,920 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:38,920 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,920 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,921 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:38,921 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,921 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,924 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 05:29:38,924 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@67264be4 05:29:38,925 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1233408902] 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#941092838] 05:29:38,926 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 05:29:38,926 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 05:29:38,926 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,926 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1233408902] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,926 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#941092838] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,926 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 05:29:38,926 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1641726133 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,926 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1641726133] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,926 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-696942370] 05:29:38,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-61009275] 05:29:38,927 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,928 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-696942370] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,928 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-61009275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,928 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-1210731678] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:38,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:38,928 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,928 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1914680332 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1914680332] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 05:29:38,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 05:29:38,929 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:29:38,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:38,929 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 05:29:38,929 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,929 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 05:29:38,929 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 05:29:38,929 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,930 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,930 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 05:29:38,930 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 05:29:38,930 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 05:29:38,930 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:29:38,930 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:38,930 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 05:29:38,930 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 05:29:38,930 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 05:29:38,930 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 05:29:38,931 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:38,931 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,932 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,932 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:38,932 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,932 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,934 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 05:29:38,935 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@71102d90 05:29:38,935 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 05:29:38,935 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 05:29:38,935 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 05:29:38,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:38,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,935 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,935 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-885573433] 05:29:38,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,936 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,936 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#533147614] 05:29:38,937 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-1=false} 05:29:38,937 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 05:29:38,937 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-885573433] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,937 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 05:29:38,937 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1626576523] 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1626576523] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1066493975 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]], state: Idle 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Returning INVALID_REQUEST for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:38,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-11] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard cars java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$11.onComplete(ShardManager.java:1558) at akka.dispatch.OnComplete.internal(Future.scala:300) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 05:29:38,938 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 05:29:38,938 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,939 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 05:29:38,939 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-395863976] 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:38,939 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1233408902] 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:38,939 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 05:29:38,939 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 05:29:38,939 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1233408902] 05:29:38,939 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,939 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 05:29:38,939 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 05:29:38,940 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,940 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 05:29:38,940 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1806f1a0 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1806f1a0 05:29:38,940 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 05:29:38,940 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@19df8add 05:29:38,940 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@19df8add 05:29:38,940 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,940 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 05:29:38,940 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 05:29:38,940 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:38,940 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 05:29:38,940 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2f35490b 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2f35490b 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,941 PM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 05:29:38,941 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 05:29:38,942 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1870304037 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,942 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#883553969] 05:29:38,942 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:38,942 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:38,942 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:38,942 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 05:29:38,942 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,942 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,942 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:38,943 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 05:29:38,943 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 05:29:38,944 PM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 05:29:38,944 PM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Aborting 0 pending queued transactions 05:29:38,944 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:38,944 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:38,945 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:38,945 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:38,946 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:38,956 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:38,969 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3727582967672589358] 05:29:38,969 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 05:29:38,971 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 05:29:38,971 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 05:29:38,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:38,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:38,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 05:29:38,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 05:29:38,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 05:29:38,991 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,992 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:38,994 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 05:29:38,998 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#331626749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,1 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,2 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,3 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f6fd3f2 05:29:39,3 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,5 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,6 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] 05:29:39,6 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,6 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,6 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,6 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1692035580 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1692035580] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#29889865] 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#29889865] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,7 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,7 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#420194690] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1584970272 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1043238755] 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1584970272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,7 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1043238755] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,8 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,8 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,8 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,8 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,8 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,8 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,8 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,10 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,11 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,11 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,11 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-274989693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,12 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 05:29:39,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#420194690] 05:29:39,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 05:29:39,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,12 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,12 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,12 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,13 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,13 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,14 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,15 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,15 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3597b882 05:29:39,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,18 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-14053351] 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-1669164899] 05:29:39,19 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,20 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,20 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,20 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1851326064 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,20 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-973055386] 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1073910875 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-14053351] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-1669164899] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1213347675] 05:29:39,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,21 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,21 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 05:29:39,21 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,21 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,22 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,22 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 05:29:39,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,22 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,22 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,22 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,22 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:39,32 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3032594824679656857] 05:29:39,32 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 05:29:39,34 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:39,34 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 05:29:39,40 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:39,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:39,46 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,47 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,49 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#656272250] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,57 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 05:29:39,64 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 05:29:39,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,69 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5dd79319 05:29:39,70 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,71 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,71 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,71 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,71 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,71 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,72 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,72 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,77 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 05:29:39,77 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,79 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,79 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,79 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,79 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,79 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,80 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1702471885 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-565526129] 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#847555149] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-712656465] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1609668082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1702471885] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,82 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-565526129] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,83 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,83 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 05:29:39,83 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,83 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 05:29:39,83 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,83 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,83 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,83 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,83 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,84 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,84 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,84 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,86 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#191996707] 05:29:39,86 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#191996707] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,87 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1617791643] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,87 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1617791643 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,88 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,88 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,88 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,91 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,92 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5da0eb9a 05:29:39,92 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,94 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,94 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-314591824] 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,95 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#534723435] 05:29:39,96 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-314591824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,96 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,97 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,97 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1458035932 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,97 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#1489555083] 05:29:39,97 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#-914828023] 05:29:39,97 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#2128857520 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,97 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#534723435] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,97 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,97 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,98 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,98 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,98 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,98 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,98 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 05:29:39,99 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,99 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,99 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,99 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,99 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,99 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,99 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,102 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 05:29:39,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2943140427537931932] 05:29:39,112 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 05:29:39,114 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:39,114 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 05:29:39,117 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 05:29:39,117 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 05:29:39,125 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,126 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,128 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,138 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-774687679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,138 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 05:29:39,140 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 05:29:39,140 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,140 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,143 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,146 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,146 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@475d17d6 05:29:39,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,147 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,147 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 05:29:39,148 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,148 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,148 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1986038412] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,148 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1396854150 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,148 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,149 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#2027872339] 05:29:39,149 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1670026925] 05:29:39,149 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1239231456 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,150 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,152 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 05:29:39,153 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 05:29:39,153 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,153 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#733863637] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-1396854150] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1239231456] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1670026925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#2027872339] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-723829168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,154 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 05:29:39,154 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,155 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,155 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 05:29:39,155 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,155 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,155 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,155 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,155 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,155 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,156 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,157 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,158 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,159 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,159 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,159 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,159 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,159 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,159 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,159 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,160 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@613281a3 05:29:39,160 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 05:29:39,160 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,160 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 05:29:39,160 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:39,160 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 05:29:39,160 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,160 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#256341420] 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#63226130] 05:29:39,161 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,162 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#256341420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,162 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-582760927] 05:29:39,162 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - akkaDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-582760927] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 05:29:39,162 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 05:29:39,162 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#855922650 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,163 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,163 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1200245492] 05:29:39,163 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1716997878 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,163 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 05:29:39,163 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,163 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,163 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 05:29:39,163 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 05:29:39,163 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,163 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,163 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 05:29:39,163 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,163 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,164 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,164 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,164 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,164 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 05:29:39,164 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 05:29:39,164 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,164 PM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,164 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 05:29:39,164 PM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,164 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 05:29:39,164 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 05:29:39,364 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:39,364 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,365 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 05:29:39,365 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 05:29:39,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:39,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,375 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 05:29:39,375 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:39,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,415 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 05:29:39,415 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 05:29:39,424 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 05:29:39,424 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 05:29:39,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:39,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,744 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 05:29:39,754 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:39,754 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,754 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 05:29:39,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:39,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,774 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 05:29:39,824 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:39,824 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,824 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 05:29:39,985 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 05:29:39,985 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - akkaMemberChanged Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-17] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,986 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:39,993 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,73 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:40,73 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,73 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to akka://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,74 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,114 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,116 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,116 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,116 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:40,118 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 05:29:40,118 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,118 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,118 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:40,118 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,119 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:40,119 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:40,119 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 05:29:40,119 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 05:29:40,119 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,119 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 05:29:40,120 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,120 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,120 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,120 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,120 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 05:29:40,120 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,120 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,120 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 05:29:40,120 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 05:29:40,121 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,121 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,121 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,121 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,121 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,121 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,121 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,121 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 05:29:40,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,130 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 05:29:40,174 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:40,174 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,175 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 05:29:40,175 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,175 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:40,178 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 05:29:40,178 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 05:29:40,178 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,178 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 05:29:40,178 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 05:29:40,179 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,179 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 05:29:40,179 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,179 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,179 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,179 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 05:29:40,179 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 05:29:40,179 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 05:29:40,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,180 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,184 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 05:29:40,184 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,184 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 05:29:40,185 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,185 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,185 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,185 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 05:29:40,185 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,185 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 05:29:40,185 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,186 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 05:29:40,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 05:29:40,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 05:29:40,186 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,186 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 05:29:40,186 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 05:29:40,186 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 05:29:40,187 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 05:29:40,187 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 05:29:40,187 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,187 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 05:29:40,187 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 05:29:40,187 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 05:29:40,187 PM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:40,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,188 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,188 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 05:29:40,188 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,188 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 05:29:40,188 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 05:29:40,188 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 05:29:40,188 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 05:29:40,188 PM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 05:29:40,188 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#331626749] 05:29:40,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:40,188 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,188 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] 05:29:40,188 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 05:29:40,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:40,189 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] 05:29:40,189 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#656272250] 05:29:40,189 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:40,189 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#847555149] 05:29:40,189 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#847555149] 05:29:40,190 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-774687679] 05:29:40,190 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:40,190 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1986038412] 05:29:40,190 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1986038412] 05:29:40,190 PM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-2=false, member-3=false} 05:29:40,190 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 05:29:40,190 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:40,191 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,191 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 05:29:40,191 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,191 PM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Not leader - forwarding to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard)] 05:29:40,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 05:29:40,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=b5bfe7a9-8cd7-48c5-ab31-df3661db339d}, logIndex: 0, isSendImmediate: true 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Setting last applied to 0 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 05:29:40,193 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] 05:29:40,194 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] has been successfully replicated to a majority of followers 05:29:40,194 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 05:29:40,194 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,195 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,195 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 05:29:40,195 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard cars 05:29:40,195 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 05:29:40,195 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 05:29:40,195 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=akka://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING]] 05:29:40,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 05:29:40,196 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 05:29:40,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#331626749] 05:29:40,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 05:29:40,197 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] 05:29:40,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1158705272] 05:29:40,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#656272250] 05:29:40,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 05:29:40,197 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#847555149] 05:29:40,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#847555149] 05:29:40,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-774687679] 05:29:40,197 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 05:29:40,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:40,198 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 05:29:40,198 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1986038412] 05:29:40,198 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1986038412] 05:29:40,200 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:40,200 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,200 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,201 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,201 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,201 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,201 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,201 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 05:29:40,201 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,201 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 05:29:40,202 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,202 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,202 PM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,202 PM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,202 PM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,202 PM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,202 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 05:29:40,202 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 05:29:40,203 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:40,203 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:40,207 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:40,207 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:40,207 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:40,207 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:40,208 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:40,221 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:40,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,222 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 05:29:40,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,222 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,222 PM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 05:29:40,222 PM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,223 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:40,223 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,223 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 05:29:40,223 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 05:29:40,223 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:40,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,224 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 05:29:40,224 PM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,224 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:40,224 PM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:40,224 PM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,225 PM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:40,225 PM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:40,225 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 05:29:40,225 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 05:29:40,226 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:40,226 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:41,5 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 05:29:41,224 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [2943140427537931932] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 05:29:41,241 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:41,241 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 05:29:41,243 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 05:29:41,245 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3032594824679656857] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [10/16/2024 17:29:41.245] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 05:29:41,247 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 05:29:41,250 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 05:29:41,251 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:41,251 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:41,251 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 05:29:41,251 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 05:29:41,251 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7aba0ae0 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7aba0ae0 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 05:29:41,251 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@668dfc6a 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@668dfc6a 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 05:29:41,252 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,252 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 05:29:41,714 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 462.8 ms 05:29:41,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 05:29:41,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 05:29:41,714 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 05:29:41,715 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 05:29:41,715 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 533.4 ms 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 05:29:41,785 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:41,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 05:29:41,786 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 05:29:41,786 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 05:29:41,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:41,786 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Initiating leader transfer 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e423ce 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e423ce 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@684a0ecb 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@684a0ecb 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 05:29:41,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 05:29:41,787 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 05:29:42,94 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 05:29:42,94 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 307.1 ms 05:29:42,94 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 05:29:42,94 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 05:29:42,94 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 05:29:42,95 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 05:29:42,95 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:42,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 05:29:42,354 PM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 567.8 ms 05:29:42,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 05:29:42,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 05:29:42,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 05:29:42,355 PM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 05:29:42,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Aborting 0 pending queued transactions 05:29:42,356 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 05:29:42,356 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 05:29:42,357 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 05:29:42,357 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 05:29:42,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [10/16/2024 17:29:42.361] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config