09:16:18,298 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:18,866 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1324885883420615496] 09:16:18,894 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:16:19,67 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:19,68 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:16:19,137 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:19,137 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:19,466 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:16:19,467 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:19,478 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:16:19,497 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:19,502 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build what is described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#0.3 WARNING: A Java agent has been loaded dynamically (/tmp/r/net/bytebuddy/byte-buddy-agent/1.15.3/byte-buddy-agent-1.15.3.jar) WARNING: If a serviceability tool is in use, please run with -XX:+EnableDynamicAgentLoading to hide this warning WARNING: If a serviceability tool is not in use, please run with -Djdk.instrument.traceUsage for more information WARNING: Dynamic loading of agents will be disallowed by default in a future release Mockito is currently self-attaching to enable the inline-mock-maker. This will no longer work in future releases of the JDK. Please add Mockito as an agent to your build what is described in Mockito's documentation: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#0.3 09:16:20,768 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 09:16:20,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:20,938 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:16:20,979 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:16:20,983 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 09:16:21,20 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 09:16:21,31 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:21,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:16:21,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:16:21,57 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 09:16:21,117 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@75338920} 09:16:21,127 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 09:16:21,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:21,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 09:16:21,136 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:16:21,136 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:16:21,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-54533452] 09:16:21,137 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 09:16:21,138 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:16:21,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:16:21,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1618673172] 09:16:21,140 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-54533452] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,151 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1618673172] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,162 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testBackupDatastore, persistent : true 09:16:21,164 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testBackupDatastore, persistent : true 09:16:21,185 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,186 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:21,187 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:16:21,238 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,239 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:21,239 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:16:21,289 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,293 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:21,293 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:16:21,294 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1905705393 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 09:16:21,294 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#227751006 created and ready for shard:member-1-shard-people-config_testBackupDatastore 09:16:21,294 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#227751006] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,295 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1905705393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,304 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-2058539703] 09:16:21,304 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/shard-member-1:snapshot-read#-2058539703] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,304 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-433491801] 09:16:21,305 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/shard-member-1:snapshot-read#-433491801] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,307 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testBackupDatastore with journal batch size 1 09:16:21,310 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testBackupDatastore with journal batch size 1 09:16:21,324 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:21,324 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:21,332 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:21,332 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:21,334 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:16:21,335 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 09:16:21,344 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,354 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:16:21,356 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 09:16:21,356 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:21,357 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-54533452] 09:16:21,359 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,360 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 09:16:21,358 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,361 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 09:16:21,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 09:16:21,364 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 09:16:21,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 09:16:21,374 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 09:16:21,378 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 09:16:21,378 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 09:16:21,378 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-config_testBackupDatastore 09:16:21,378 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-config_testBackupDatastore 09:16:21,382 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 09:16:21,383 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:21,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testBackupDatastore is no longer the leader 09:16:21,383 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 09:16:21,383 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 09:16:21,383 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:21,384 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,384 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testBackupDatastore is no longer the leader 09:16:21,384 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 09:16:21,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate): Received ElectionTimeout 09:16:21,409 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Candidate): Received ElectionTimeout 09:16:21,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 09:16:21,413 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:21,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:16:21,414 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore (Leader): Election: Leader has following peers: [] 09:16:21,414 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:21,414 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:16:21,417 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 09:16:21,417 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 09:16:21,417 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 09:16:21,417 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 09:16:21,424 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 09:16:21,424 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 09:16:21,425 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testBackupDatastore, leaderId=member-1-shard-people-config_testBackupDatastore, leaderPayloadVersion=13] 09:16:21,428 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testBackupDatastore, leaderId=member-1-shard-cars-config_testBackupDatastore, leaderPayloadVersion=13] 09:16:21,429 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 09:16:21,429 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 09:16:21,429 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 09:16:21,474 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,475 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:16:21,475 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#1618673172] 09:16:21,475 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:21,484 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:21,494 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 09:16:21,498 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1749730988] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,501 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:16:21,503 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:16:21,503 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 09:16:21,504 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 09:16:21,504 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:21,504 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:16:21,504 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:16:21,505 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 09:16:21,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1bbbd9f6} 09:16:21,507 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 09:16:21,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:21,508 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 09:16:21,509 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:16:21,509 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:16:21,509 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#772608263] 09:16:21,509 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 09:16:21,509 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testBackupDatastore, persistent : true 09:16:21,510 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:16:21,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:16:21,510 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-217793997] 09:16:21,510 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-oper_testBackupDatastore, persistent : true 09:16:21,511 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testBackupDatastore with journal batch size 1 09:16:21,512 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testBackupDatastore with journal batch size 1 09:16:21,513 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#984122126 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 09:16:21,513 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/shard-member-1:snapshot-read#214743995] 09:16:21,514 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#772608263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,514 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/shard-member-1:snapshot-read#-1423940517] 09:16:21,514 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#-1615948999 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 09:16:21,515 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:21,516 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:16:21,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 09:16:21,516 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,516 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 09:16:21,516 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 09:16:21,516 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-people-oper_testBackupDatastore 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,517 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testBackupDatastore is no longer the leader 09:16:21,517 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 09:16:21,518 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate): Received ElectionTimeout 09:16:21,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 09:16:21,519 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:21,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:16:21,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 09:16:21,519 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 09:16:21,520 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 09:16:21,520 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:21,520 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testBackupDatastore, leaderId=member-1-shard-people-oper_testBackupDatastore, leaderPayloadVersion=13] 09:16:21,520 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:21,520 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 09:16:21,520 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,520 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:16:21,520 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 09:16:21,520 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 09:16:21,520 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 09:16:21,520 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower): Received TimeoutNow - switching to Candidate 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Election: Candidate has following voting peers: [] 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testBackupDatastore 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:16:21,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testBackupDatastore is no longer the leader 09:16:21,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 09:16:21,522 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate): Received ElectionTimeout 09:16:21,522 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore (Leader): Election: Leader has following peers: [] 09:16:21,522 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:21,522 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:16:21,522 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 09:16:21,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 09:16:21,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 09:16:21,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testBackupDatastore, leaderId=member-1-shard-cars-oper_testBackupDatastore, leaderPayloadVersion=13] 09:16:21,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 09:16:21,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 09:16:21,533 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-217793997] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:21,558 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1749730988] 09:16:21,558 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:21,559 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#772608263] 09:16:21,609 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-1749730988] 09:16:21,610 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:16:21,610 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-217793997] 09:16:21,625 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 09:16:21,627 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 09:16:21,628 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 09:16:21,631 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 09:16:21,631 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 09:16:21,631 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,631 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 09:16:21,632 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onGetSnapshot 09:16:21,632 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,632 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 09:16:21,632 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 09:16:21,633 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,647 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,647 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,647 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,647 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,651 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@683d5515, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,651 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@35979c62, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,661 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2fcd2cdc, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,661 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@9062f5, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,661 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 09:16:21,661 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 09:16:21,701 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 09:16:21,738 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1211564749] 09:16:21,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@37c52ab8 09:16:21,740 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:21,741 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#-54533452] 09:16:21,770 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testBackupDatastore 09:16:21,776 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 09:16:21,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 09:16:21,777 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 09:16:21,777 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 09:16:21,778 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 09:16:21,778 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onGetSnapshot 09:16:21,778 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,778 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onGetSnapshot 09:16:21,778 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,779 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,779 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,779 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2343da6, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,780 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply [id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2afb974, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,780 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 09:16:21,781 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onGetSnapshot 09:16:21,781 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:21,781 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:21,782 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply [id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3c1095e2, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 09:16:21,995 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 09:16:21,997 AM [cluster-test-pekko.actor.default-dispatcher-5] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:16:22,4 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 09:16:22,6 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 09:16:22,8 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 09:16:22,9 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onShutDown 09:16:22,10 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:16:22,11 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@481d9b4f 09:16:22,11 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@481d9b4f 09:16:22,12 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-config_testBackupDatastore 09:16:22,12 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:16:22,13 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:16:22,16 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 09:16:22,17 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 09:16:22,17 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 09:16:22,17 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:16:22,17 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onShutDown 09:16:22,17 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3b79701f 09:16:22,17 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3b79701f 09:16:22,18 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onShutDown 09:16:22,18 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2a358169 09:16:22,18 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2a358169 09:16:22,19 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testBackupDatastore 09:16:22,19 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testBackupDatastore 09:16:22,20 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:16:22,21 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:16:22,26 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:22,28 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:22,52 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:16:22,105 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:22,132 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4095621115103119567] 09:16:22,135 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:16:22,140 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:22,140 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:16:22,148 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:22,148 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:22,183 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:16:22,183 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:22,183 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:16:22,187 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,192 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,199 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 09:16:22,217 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1443909367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,218 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:16:22,227 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:16:22,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,228 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:22,228 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,228 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,232 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:16:22,234 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 09:16:22,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@73dcc2e9} 09:16:22,236 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:22,236 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,240 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,240 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,240 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939] 09:16:22,240 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,240 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddReplicasForAllShards, persistent : true 09:16:22,242 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,245 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-2018084792 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,245 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#194125098] 09:16:22,246 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-2018084792] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,246 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 09:16:22,247 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testAddReplicasForAllShards, persistent : true 09:16:22,248 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1628705512 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,249 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1498068119] 09:16:22,250 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277] 09:16:22,250 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#194125098] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:22,250 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,250 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:22,250 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1628705512] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,251 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#1498068119] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,252 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:16:22,252 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 09:16:22,256 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddReplicasForAllShards 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:22,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,257 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddReplicasForAllShards is no longer the leader 09:16:22,257 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:16:22,257 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 09:16:22,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 09:16:22,258 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:22,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:22,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:16:22,258 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:16:22,258 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:22,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:22,259 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 09:16:22,260 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testAddReplicasForAllShards with journal batch size 1 09:16:22,262 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:22,263 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:22,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:16:22,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 09:16:22,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 09:16:22,263 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:22,263 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testAddReplicasForAllShards 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,264 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testAddReplicasForAllShards is no longer the leader 09:16:22,265 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:16:22,265 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 09:16:22,266 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 09:16:22,266 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:22,266 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:22,266 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:16:22,267 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:16:22,267 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:22,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:22,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 09:16:22,268 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:22,286 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1443909367] 09:16:22,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:22,287 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939] 09:16:22,338 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1443909367] 09:16:22,338 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:16:22,338 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277] 09:16:22,339 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,345 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,352 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 09:16:22,355 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1726105935] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,356 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:16:22,358 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:16:22,358 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,359 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,359 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:22,359 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,359 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,361 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:16:22,362 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@eea5eea} 09:16:22,362 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 09:16:22,363 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:22,363 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,365 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddReplicasForAllShards, persistent : true 09:16:22,365 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 09:16:22,366 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,366 AM [cluster-test-pekko.actor.default-dispatcher-7] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#-937748300] 09:16:22,366 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#-23920650 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,367 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#-23920650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,367 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,367 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,367 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402] 09:16:22,367 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,368 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,368 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,368 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1770997901] 09:16:22,369 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:22,369 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:22,369 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,369 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 09:16:22,369 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 09:16:22,370 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:16:22,370 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 09:16:22,370 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-oper_testAddReplicasForAllShards, persistent : true 09:16:22,371 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-2033781003 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,371 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 09:16:22,371 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/shard-member-1:snapshot-read#1917941228] 09:16:22,371 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 09:16:22,371 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 09:16:22,372 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:22,372 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:16:22,372 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:22,372 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,372 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 09:16:22,372 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:22,372 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:22,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:16:22,372 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 09:16:22,372 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 09:16:22,372 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 09:16:22,372 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testAddReplicasForAllShards 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,373 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testAddReplicasForAllShards is no longer the leader 09:16:22,373 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 09:16:22,374 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 09:16:22,374 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 09:16:22,374 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 09:16:22,374 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:22,374 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:22,374 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 09:16:22,375 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 09:16:22,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 09:16:22,375 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:16:22,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 09:16:22,375 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:22,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:22,375 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:22,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 09:16:22,375 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 09:16:22,375 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 09:16:22,375 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:16:22,375 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:22,376 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 09:16:22,376 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:16:22,422 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1726105935] 09:16:22,423 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:22,423 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402] 09:16:22,474 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1726105935] 09:16:22,474 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:16:22,474 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1770997901] 09:16:22,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5e1e91d8] 09:16:22,495 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 09:16:22,498 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-pets-config_testAddReplicasForAllShards, persistent : true 09:16:22,500 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/shard-member-1:snapshot-read#-502769676] 09:16:22,500 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-1545505205 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 09:16:22,500 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 09:16:22,502 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:22,502 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:22,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 09:16:22,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 09:16:22,503 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,503 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 09:16:22,503 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:22,503 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower): Received TimeoutNow - switching to Candidate 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Election: Candidate has following voting peers: [] 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=member-1-shard-pets-config_testAddReplicasForAllShards 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:22,504 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testAddReplicasForAllShards is no longer the leader 09:16:22,504 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:16:22,506 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Received ElectionTimeout 09:16:22,506 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Election: Leader has following peers: [] 09:16:22,506 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:22,506 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:22,506 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:16:22,506 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:16:22,506 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:22,507 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:22,507 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 09:16:22,507 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:22,552 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-1443909367] 09:16:22,553 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:16:22,553 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-1527924044] 09:16:22,581 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:16:22,602 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-7738791957976036651] 09:16:22,603 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:16:22,610 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:22,610 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:16:22,611 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:22,611 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:22,638 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,642 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,646 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 09:16:22,658 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2025567083] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,663 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:16:22,668 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:16:22,670 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:16:22,671 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 09:16:22,671 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7ebbeda9} 09:16:22,672 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:22,676 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:22,684 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 09:16:22,686 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#797949117] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:22,687 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:16:22,691 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:16:22,693 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:16:22,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@707c0c0e} 09:16:22,694 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 09:16:22,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:22,976 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:16:23,23 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:23,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:23,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:23,28 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:16:23,160 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:16:23,160 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:23,160 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:23,160 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,160 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:16:23,161 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,208 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@3dbdc06b] 09:16:23,210 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 09:16:23,211 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-pets-config_testAddReplicasForAllShards, persistent : true 09:16:23,211 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@22603f97] 09:16:23,211 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#174426905] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#717507023 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#717507023] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,212 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testAddReplicasForAllShards with journal batch size 1 09:16:23,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#258392049] 09:16:23,213 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#258392049] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,213 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 09:16:23,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,215 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,215 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 09:16:23,215 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,215 AM [cluster-test-shard-dispatcher-11] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:16:23,216 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 09:16:23,216 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 09:16:23,216 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:23,216 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 09:16:23,220 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-no-leader-oper_testAddReplicasForAllShards, persistent : true 09:16:23,221 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1930870177 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 09:16:23,221 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-2095062551] 09:16:23,221 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-no-leader-oper_testAddReplicasForAllShards with journal batch size 1 09:16:23,222 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,222 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,222 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 09:16:23,222 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 09:16:23,223 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,223 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is no longer the leader 09:16:23,223 AM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:16:23,223 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 09:16:23,223 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,223 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:16:23,223 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 09:16:23,224 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:23,224 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 09:16:23,225 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 09:16:23,226 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-2033175492] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,227 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1930870177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,227 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#-2095062551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 09:16:23,227 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:23,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 09:16:23,227 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 09:16:23,228 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 09:16:23,227 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:23,228 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:23,228 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:16:23,230 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:16:23,231 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:16:23,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 09:16:23,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:16:23,234 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:16:23,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:16:23,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:16:23,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 09:16:23,236 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:16:23,239 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:23,241 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939, primaryVersion=13] 09:16:23,241 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 09:16:23,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277, primaryVersion=13] 09:16:23,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 09:16:23,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-1527924044, primaryVersion=13] 09:16:23,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:23,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402, primaryVersion=13] 09:16:23,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 09:16:23,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1770997901, primaryVersion=13] 09:16:23,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 09:16:23,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 09:16:23,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939 for shard member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,251 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277 for shard member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,252 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 09:16:23,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-1527924044 for shard member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,254 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1760547806] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,255 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testAddReplicasForAllShards, persistent : true 09:16:23,256 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#53107938 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,256 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#53107938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:23,256 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#-1845498874] 09:16:23,257 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testAddReplicasForAllShards with journal batch size 1 09:16:23,258 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddReplicasForAllShards, persistent : true 09:16:23,259 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,260 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#444534033 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,260 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1496051964] 09:16:23,261 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:23,261 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddReplicasForAllShards with journal batch size 1 09:16:23,261 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,261 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,262 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 09:16:23,262 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,262 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:23,262 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 09:16:23,262 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,262 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:23,262 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddReplicasForAllShards, persistent : true 09:16:23,263 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,263 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#-1853229833 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,264 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/shard-member-2:snapshot-read#1516212250] 09:16:23,264 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:16:23,264 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 09:16:23,265 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,265 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,265 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,265 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 09:16:23,265 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 09:16:23,265 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,265 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:23,266 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,266 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddReplicasForAllShards with journal batch size 1 09:16:23,266 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1770997901 for shard member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,267 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1057) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:213) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:16:23,268 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,268 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:16:23,267 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-oper_testAddReplicasForAllShards, persistent : true 09:16:23,268 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] ClusterAdminRpcService - Failed to add replica for shard no-leader, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1057) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:213) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:16:23,268 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 09:16:23,268 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,269 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:16:23,269 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,269 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 09:16:23,269 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testAddReplicasForAllShards with journal batch size 1 09:16:23,269 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:16:23,269 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/shard-member-2:snapshot-read#1995143256] 09:16:23,269 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 09:16:23,271 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 09:16:23,271 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#213338415 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,269 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 09:16:23,273 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onAddServer: AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true], state: Idle 09:16:23,273 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,273 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:23,273 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:23,273 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:23,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:16:23,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 09:16:23,274 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,275 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 09:16:23,277 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,277 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:23,277 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,277 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 09:16:23,277 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:16:23,276 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,274 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,273 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 09:16:23,275 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,279 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 09:16:23,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:23,279 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:23,273 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 09:16:23,280 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 09:16:23,279 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: lastSequenceNumber prior to capture: 1 09:16:23,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:23,280 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:23,280 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:23,281 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 09:16:23,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:16:23,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 09:16:23,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,282 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 09:16:23,282 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2bc0efc8, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 09:16:23,282 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:23,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 09:16:23,283 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@323a5c88, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddReplicasForAllShards, ServerConfigPayload=null] 09:16:23,283 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:23,283 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 09:16:23,284 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:16:23,284 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 09:16:23,284 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,284 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 09:16:23,284 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3f01b762, electionTerm=1, electionVotedFor=member-1-shard-pets-config_testAddReplicasForAllShards, ServerConfigPayload=null] 09:16:23,284 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:23,284 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,284 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 09:16:23,284 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@21f8fb7b, electionTerm=1, electionVotedFor=member-1-shard-people-config_testAddReplicasForAllShards, ServerConfigPayload=null] 09:16:23,284 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:23,284 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,284 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: CaptureSnapshotReply received by actor 09:16:23,285 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6edf89d0, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddReplicasForAllShards, ServerConfigPayload=null] 09:16:23,285 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:23,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 09:16:23,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendInstallSnapshot 09:16:23,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 09:16:23,286 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 09:16:23,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-config_testAddReplicasForAllShards: 229 09:16:23,287 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-pets-config_testAddReplicasForAllShards: 229 09:16:23,287 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddReplicasForAllShards: 229 09:16:23,288 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-people-config_testAddReplicasForAllShards), Chunk: 1/1 09:16:23,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-pets-config_testAddReplicasForAllShards), Chunk: 1/1 09:16:23,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendInstallSnapshot 09:16:23,288 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,288 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddReplicasForAllShards: 229 09:16:23,289 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddReplicasForAllShards, member-2-shard-cars-config_testAddReplicasForAllShards), Chunk: 1/1 09:16:23,289 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,289 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,289 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,288 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-cars-oper_testAddReplicasForAllShards), Chunk: 1/1 09:16:23,288 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,290 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 09:16:23,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,293 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: handled a journal response 09:16:23,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): next snapshot chunk size for follower member-2-shard-people-oper_testAddReplicasForAllShards: 229 09:16:23,293 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: handled a journal response 09:16:23,293 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 09:16:23,293 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: handled a journal response 09:16:23,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddReplicasForAllShards, member-2-shard-people-oper_testAddReplicasForAllShards), Chunk: 1/1 09:16:23,293 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a journal response 09:16:23,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 09:16:23,302 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:23,306 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,300 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:23,306 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,301 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:23,306 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,300 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:23,306 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:23,306 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:23,306 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,302 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:23,308 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,308 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:23,306 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:23,308 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:23,308 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:23,308 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:23,308 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:23,308 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-config_testAddReplicasForAllShards 09:16:23,308 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:23,308 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-pets-config_testAddReplicasForAllShards 09:16:23,308 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:23,308 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:23,309 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-people-oper_testAddReplicasForAllShards 09:16:23,309 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddReplicasForAllShards 09:16:23,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 09:16:23,310 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 09:16:23,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 09:16:23,311 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=-1] 09:16:23,311 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,311 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@35702b7b, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:23,311 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:23,311 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 09:16:23,312 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,312 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,312 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 09:16:23,312 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ba88149, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1bf2cd6e, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:23,312 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 09:16:23,312 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 09:16:23,312 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@f193a09, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:23,313 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:23,313 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=-1] 09:16:23,313 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 09:16:23,313 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,313 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 09:16:23,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to -1 09:16:23,314 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 09:16:23,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,314 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 09:16:23,314 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 09:16:23,314 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,314 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:23,314 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:23,314 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 09:16:23,314 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:23,316 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2121f426, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:23,316 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:23,316 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Snapshot success - sequence number: 1 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to -1 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Set currentTerm=1, votedFor=null 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 09:16:23,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:23,331 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#1416018505] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 09:16:23,331 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1544369052] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 09:16:23,331 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1874963676] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 09:16:23,332 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 19.50 ms 09:16:23,332 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 18.40 ms 09:16:23,332 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 18.26 ms 09:16:23,333 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 09:16:23,333 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 09:16:23,333 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 09:16:23,333 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1748924782] 09:16:23,333 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#1416018505] 09:16:23,334 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1874963676] 09:16:23,334 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1544369052] 09:16:23,334 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#1748924782] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 09:16:23,334 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 20.33 ms 09:16:23,334 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 09:16:23,334 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:16:23,335 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:16:23,335 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 09:16:23,335 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:16:23,333 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,333 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,333 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,337 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,337 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,337 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a journal response 09:16:23,337 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a journal response 09:16:23,337 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,337 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a journal response 09:16:23,338 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 3.699 ms 09:16:23,341 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,341 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,341 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: handled a snapshot response 09:16:23,342 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 6.912 ms 09:16:23,342 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 7.120 ms 09:16:23,342 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 7.433 ms 09:16:23,334 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a snapshot response 09:16:23,343 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: handled a journal response 09:16:23,343 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,343 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:23,344 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:23,344 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#-2064721443] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 09:16:23,344 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 27.46 ms 09:16:23,344 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 09:16:23,344 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-pets-config_testAddReplicasForAllShards] 09:16:23,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,344 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:23,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:23,344 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#-2064721443] 09:16:23,344 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,344 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 09:16:23,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,345 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:23,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:23,344 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-config_testAddReplicasForAllShards] 09:16:23,345 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:23,345 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:23,345 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-people-oper_testAddReplicasForAllShards] 09:16:23,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddReplicasForAllShards] 09:16:23,345 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 119.5 μs 09:16:23,345 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]] 09:16:23,345 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]] 09:16:23,345 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]] 09:16:23,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]] 09:16:23,348 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: SnapshotComplete received 09:16:23,348 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a snapshot response 09:16:23,348 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: handled a journal response 09:16:23,348 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, chunkIndex=1, success=true] 09:16:23,349 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:23,349 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:23,349 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddReplicasForAllShards] 09:16:23,349 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]] 09:16:23,348 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,349 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,348 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,352 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,352 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,352 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,351 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,352 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,354 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=be88d158-947e-41a2-b51e-91adacdb141b}, logIndex: 0, isSendImmediate: true 09:16:23,354 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=adb49d36-3066-4344-9816-cd2b170c6112}, logIndex: 0, isSendImmediate: true 09:16:23,354 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f1d86a63-8dfc-4351-bb5f-9c2962764305}, logIndex: 0, isSendImmediate: true 09:16:23,354 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=5bd24015-2bed-42da-8959-ca2cbeb8a08e}, logIndex: 0, isSendImmediate: true 09:16:23,354 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=dd37ba22-0064-4ce7-9998-56889e9cd49d}, logIndex: 0, isSendImmediate: true 09:16:23,355 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,356 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,356 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,356 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,356 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,356 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,356 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,356 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,356 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,356 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,356 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,356 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,356 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,364 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,364 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-pets-config_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,356 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-people-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,367 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,368 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,367 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,368 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,356 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddReplicasForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,371 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddReplicasForAllShards, newServerAddress=null, votingMember=true] 09:16:23,374 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,376 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,376 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 09:16:23,376 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:23,376 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#378702277 09:16:23,376 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 09:16:23,377 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,377 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,377 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,377 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 09:16:23,377 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,377 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 09:16:23,377 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,377 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 09:16:23,377 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:23,378 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#-1770997901 09:16:23,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 09:16:23,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [people] 09:16:23,378 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:16:23,378 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 09:16:23,377 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,378 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:16:23,378 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:23,378 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 09:16:23,378 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,378 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:23,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:23,378 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 09:16:23,378 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,378 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:23,378 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,379 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,379 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,379 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards, votingState=VOTING]] 09:16:23,379 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Log size is now 1 09:16:23,379 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:16:23,379 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards, votingState=VOTING]] 09:16:23,379 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Log size is now 1 09:16:23,379 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 09:16:23,379 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testAddReplicasForAllShards, address=null, votingState=VOTING]] 09:16:23,380 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Log size is now 1 09:16:23,380 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-444940939 09:16:23,380 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:16:23,381 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,381 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars, people] 09:16:23,381 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:16:23,381 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 09:16:23,381 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:23,382 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:16:23,382 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards, votingState=VOTING]] 09:16:23,382 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Log size is now 1 09:16:23,379 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:23,379 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:23,383 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testAddReplicasForAllShards, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:23,383 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddReplicasForAllShards, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:23,382 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-people-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:23,384 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,384 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,384 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testAddReplicasForAllShards, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:23,384 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,385 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:23,385 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-oper_testAddReplicasForAllShards 09:16:23,385 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,385 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-1527924044 09:16:23,385 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:16:23,385 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testAddReplicasForAllShards is no longer the leader 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]]]] 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Number of entries to be appended = 1 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddReplicasForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards, votingState=VOTING]] 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Log size is now 1 09:16:23,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:23,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:23,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddReplicasForAllShards 09:16:23,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:16:23,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddReplicasForAllShards is no longer the leader 09:16:23,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,385 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 09:16:23,385 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 09:16:23,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,386 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:23,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-people-config_testAddReplicasForAllShards 09:16:23,387 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,386 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddReplicasForAllShards, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:23,387 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 09:16:23,387 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 09:16:23,387 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 09:16:23,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:16:23,380 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testAddReplicasForAllShards, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:23,387 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testAddReplicasForAllShards is no longer the leader 09:16:23,387 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:23,387 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#813925402 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:16:23,388 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testAddReplicasForAllShards, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderPayloadVersion=13] 09:16:23,388 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 09:16:23,388 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,388 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]]] 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 09:16:23,388 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 09:16:23,388 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:23,388 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:16:23,388 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 09:16:23,388 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:16:23,389 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddReplicasForAllShards is no longer the leader 09:16:23,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:16:23,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:23,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 09:16:23,390 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2025567083] 09:16:23,391 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:23,388 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Re-initializing to Follower with previous leaderId member-1-shard-pets-config_testAddReplicasForAllShards 09:16:23,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:16:23,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testAddReplicasForAllShards is no longer the leader 09:16:23,392 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 09:16:23,393 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#407654595] 09:16:23,393 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#407654595] 09:16:23,394 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2025567083] 09:16:23,394 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:16:23,394 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1760547806] 09:16:23,394 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1760547806] 09:16:23,395 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2025567083] 09:16:23,395 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:16:23,395 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#174426905] 09:16:23,395 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#174426905] 09:16:23,395 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 09:16:23,395 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:23,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 09:16:23,395 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:23,395 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 09:16:23,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,396 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#797949117] 09:16:23,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,396 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,396 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,396 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,396 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:23,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 09:16:23,397 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:23,397 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:16:23,397 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 09:16:23,397 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1410010203] 09:16:23,397 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#1410010203] 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Setting last applied to 0 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Moving last applied index from -1 to 0 09:16:23,397 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#797949117] 09:16:23,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,398 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:16:23,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testAddReplicasForAllShards updated: matchIndex: 0, nextIndex: 1 09:16:23,398 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#859454728] 09:16:23,398 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#859454728] 09:16:23,398 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:23,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Setting last applied to 0 09:16:23,398 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:23,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Moving last applied index from -1 to 0 09:16:23,398 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 09:16:23,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] 09:16:23,398 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:16:23,399 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,399 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onShutDown 09:16:23,399 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onShutDown 09:16:23,400 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddReplicasForAllShards 09:16:23,400 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testAddReplicasForAllShards 09:16:23,400 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,400 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 09:16:23,400 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testAddReplicasForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:23,400 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Persisting ApplyJournalEntries with index=0 09:16:23,401 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onShutDown 09:16:23,406 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-2-shard-people-config_testAddReplicasForAllShards 09:16:23,412 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:16:23,412 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:16:23,414 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:16:23,414 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,414 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 09:16:23,415 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 09:16:23,415 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onShutDown 09:16:23,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onShutDown 09:16:23,415 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onShutDown 09:16:23,415 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddReplicasForAllShards 09:16:23,416 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 09:16:23,416 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testAddReplicasForAllShards 09:16:23,416 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:16:23,417 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:16:23,420 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:23,420 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:23,442 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:23,442 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:23,448 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:16:23,467 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:16:23,468 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 09:16:23,469 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 09:16:23,469 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 09:16:23,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onShutDown 09:16:23,469 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onShutDown 09:16:23,469 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 09:16:23,469 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onShutDown 09:16:23,470 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating leader transfer 09:16:23,470 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Initiating leader transfer 09:16:23,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating leader transfer 09:16:23,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 09:16:23,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 09:16:23,472 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 09:16:23,473 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8d91294 09:16:23,473 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b75088b 09:16:23,473 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8d91294 09:16:23,473 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b75088b 09:16:23,473 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:16:23,473 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:16:23,473 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 09:16:23,473 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 09:16:23,474 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c4dde7c 09:16:23,474 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c4dde7c 09:16:23,474 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:16:23,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,474 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:23,474 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 09:16:23,474 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:24,448 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-7738791957976036651] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:27,748 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:28,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:29,29 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:29,279 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:16:29,281 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:29,282 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:29,286 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:29,319 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:29,988 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:30,275 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:30,338 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:30,631 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:30,946 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:31,268 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:31,355 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:31,589 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:31,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:32,376 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:32,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 09:16:32,536 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 09:16:32,536 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.065 s 09:16:32,536 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 09:16:32,536 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.065 s 09:16:32,536 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 09:16:32,537 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 09:16:32,537 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 09:16:32,537 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 09:16:32,537 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 09:16:32,537 AM [cluster-test-shard-dispatcher-16] [WARN] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,537 AM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,536 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 09:16:32,537 AM [cluster-test-shard-dispatcher-15] [WARN] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,537 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.066 s 09:16:32,537 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 09:16:32,538 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 09:16:32,538 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Leadership transfer expired 09:16:32,538 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testAddReplicasForAllShards] 09:16:32,538 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testAddReplicasForAllShards] 09:16:32,539 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:32,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:32,539 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:16:32,539 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testAddReplicasForAllShards 09:16:32,539 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:16:32,539 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:32,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:32,539 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:16:32,539 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:16:32,539 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,539 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,540 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddReplicasForAllShards] 09:16:32,540 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:32,540 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:16:32,540 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:32,540 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-config_testAddReplicasForAllShards 09:16:32,541 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddReplicasForAllShards 09:16:32,542 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:16:32,542 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:16:32,543 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:16:32,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:32,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 09:16:32,544 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:16:32,544 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onShutDown 09:16:32,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onShutDown 09:16:32,544 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating leader transfer 09:16:32,545 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddReplicasForAllShards 09:16:32,545 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testAddReplicasForAllShards, leaderId=null, leaderPayloadVersion=13] 09:16:32,545 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@229c03bd 09:16:32,546 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@229c03bd 09:16:32,546 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:16:32,546 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Attempting to transfer leadership 09:16:32,546 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testAddReplicasForAllShards] 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testAddReplicasForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: leader transfer aborted 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: Failed to transfer leadership in 2.316 ms 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In unpauseLeader 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: leader transfer failed - sending PoisonPill 09:16:32,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:16:32,547 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:16:32,547 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:16:32,548 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testAddReplicasForAllShards 09:16:32,548 AM [cluster-test-pekko.actor.default-dispatcher-7] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:16:32,548 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:16:32,549 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:32,549 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:32,554 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:16:32.557] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:32,592 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:32,609 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [151208637360861784] 09:16:32,611 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:16:32,615 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:32,615 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:16:32,622 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:32,622 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:32,641 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:16:32,641 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:32,641 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:16:32,642 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,645 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,648 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:16:32,658 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,659 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:16:32,668 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:16:32,668 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,669 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,669 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:32,669 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,669 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,704 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:32,705 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:16:32,705 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3b808664} 09:16:32,706 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,706 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,706 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,709 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,713 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:16:32,714 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,714 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,715 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:32,715 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,715 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,717 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:16:32,718 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,718 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,718 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] 09:16:32,718 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStates, persistent : true 09:16:32,718 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,719 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,719 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#519372143] 09:16:32,719 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1660119292] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,719 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,719 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1660119292 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,719 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStates, persistent : true 09:16:32,720 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:16:32,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,720 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:32,720 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,721 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:32,721 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,721 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,721 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1693108232 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,721 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#419537338] 09:16:32,722 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,729 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:16:32,729 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@676455f} 09:16:32,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,731 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,731 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#519372143] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,731 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1693108232] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,731 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/shard-member-1:snapshot-read#419537338] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,732 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:32,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,732 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,733 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStates, persistent : true 09:16:32,733 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,733 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,733 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1098237329] 09:16:32,733 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-451551935 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,734 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-451551935] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,734 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#1350081509] 09:16:32,734 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStates, persistent : true 09:16:32,734 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,735 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#-2031111369 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,736 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,737 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:16:32,737 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 09:16:32,737 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,737 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:32,737 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,737 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/shard-member-1:snapshot-read#1654947099] 09:16:32,738 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,738 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:16:32,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,739 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:32,739 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,739 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,739 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,739 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,740 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:16:32,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 09:16:32,741 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,741 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:16:32,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 09:16:32,742 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,742 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,742 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,742 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,742 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:16:32,742 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:32,756 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-2124824066484412623] 09:16:32,757 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:16:32,760 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:32,760 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:16:32,762 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:32,762 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:32,782 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,785 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,787 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:16:32,796 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1233245020] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,797 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:16:32,804 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:16:32,804 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 09:16:32,805 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 09:16:32,806 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:32,809 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:16:32,810 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@91c9b37} 09:16:32,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 09:16:32,813 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,816 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:16:32,820 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 09:16:32,822 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStates, persistent : true 09:16:32,823 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,823 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1477169683] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,823 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1281601066 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 09:16:32,823 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStates, persistent : true 09:16:32,828 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:16:32,837 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#285650901] 09:16:32,839 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:16:32,840 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-11355367] 09:16:32,840 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:32,840 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,841 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,842 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,842 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:16:32,842 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 09:16:32,842 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:32,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:32,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@54582813} 09:16:32,844 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:16:32,844 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1112674289 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 09:16:32,844 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,845 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:32,845 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 09:16:32,846 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStates, persistent : true 09:16:32,847 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#135080783 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:32,847 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#722424283] 09:16:32,848 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,848 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 09:16:32,848 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStates, persistent : true 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-247585903 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/shard-member-2:snapshot-read#-1201617098] 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1281601066] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1629373211] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1132111738] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#285650901] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/shard-member-2:snapshot-read#-11355367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#1112674289] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1299530024] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#135080783] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING]] 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,850 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:16:32,850 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,850 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,850 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,850 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:16:32,851 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,852 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,853 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:16:32,853 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 09:16:32,853 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,853 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:32,853 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,853 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,853 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,859 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,860 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,861 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,861 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,861 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,861 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:16:32,861 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 09:16:32,861 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,862 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:16:32,870 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:16:32,873 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:16:32,875 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:32,875 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:32,890 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [2064743092203294317] 09:16:32,890 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:16:32,895 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:32,895 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:16:32,898 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:32,898 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:32,910 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,912 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,920 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:16:32,934 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:16:32,936 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1393729446] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,939 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:16:32,940 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 09:16:32,940 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 09:16:32,941 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:32,943 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@676ba4fb} 09:16:32,943 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:16:32,943 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,943 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 09:16:32,943 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,945 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:32,952 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 09:16:32,954 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStates, persistent : true 09:16:32,954 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:16:32,955 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,956 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStates, persistent : true 09:16:32,956 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStates with journal batch size 1 09:16:32,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2046509211] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-454144234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#928358097] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#597941852 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 09:16:32,958 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#597941852] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,959 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:16:32,972 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-597685924] 09:16:32,973 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-597685924] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,980 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-60865121] 09:16:32,980 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/shard-member-3:snapshot-read#-60865121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,983 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,983 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 294 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,983 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,984 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:16:32,984 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 09:16:32,984 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 09:16:32,984 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,984 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:32,984 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,984 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,985 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-49183296 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,985 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,985 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,985 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-49183296] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,985 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 09:16:32,985 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:16:32,985 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,985 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:32,986 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:32,986 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,986 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:16:32,986 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:16:32,987 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@19f38294} 09:16:32,987 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:16:32,988 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:32,988 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:32,988 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:32,989 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 09:16:32,990 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStates, persistent : true 09:16:32,990 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1354733790] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,990 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStates, persistent : true 09:16:32,990 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1422143422] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:32,991 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#1770376375] 09:16:32,991 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/shard-member-3:snapshot-read#700365145] 09:16:32,991 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#2010686958 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 09:16:32,991 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#504872235 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:32,991 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,991 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStates with journal batch size 1 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 288 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=1, votedFor=null 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Received ReplicatedLogEntry for recovery: index: 0, size: 282 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=null, votingState=VOTING]] 09:16:32,993 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,993 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:32,993 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,993 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:32,994 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,994 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:16:32,994 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 09:16:32,994 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,994 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:16:32,994 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:32,994 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:32,994 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:16:32,994 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:16:32,994 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:16:32,997 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,2 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:33,6 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,6 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,6 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,636 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:16:33,636 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,639 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,639 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,640 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,640 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,640 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,640 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,655 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,655 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,663 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,663 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:33,777 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 09:16:33,778 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 09:16:33,778 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,778 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:16:33,778 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,782 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:16:33,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,782 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:33,782 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,782 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,791 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:16:33,795 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:33,796 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,796 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:16:33,796 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,797 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:16:33,797 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,797 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,797 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,797 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:33,799 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:16:33,799 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:16:33,799 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,800 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:33,800 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 09:16:33,801 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,801 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,802 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:16:33,802 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,803 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,803 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:16:33,803 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:16:33,803 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:16:33,803 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:33,803 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:16:33,803 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:16:33,803 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:16:33,803 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 09:16:33,803 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,804 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,804 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:16:33,804 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,804 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,805 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:16:33,805 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,805 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,805 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 09:16:33,806 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,807 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,807 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:16:33,807 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 09:16:33,807 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,807 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,807 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:16:33,808 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:16:33,808 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,810 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,814 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,814 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,814 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:33,814 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 09:16:33,814 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:33,814 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,817 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,817 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,818 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:16:33,818 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:16:33,818 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 09:16:33,818 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,818 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,818 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 09:16:33,819 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 09:16:33,819 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,819 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 09:16:33,820 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 09:16:33,821 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,821 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,822 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,822 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,822 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,822 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,822 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,822 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 09:16:33,823 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,823 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,823 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:33,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,825 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:33,825 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,825 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,824 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,826 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,825 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received ElectionTimeout - switching to Candidate 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,826 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,826 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:33,823 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,826 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,827 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,827 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,827 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 09:16:33,827 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,827 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,827 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,826 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,828 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 09:16:33,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,829 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,829 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:16:33,829 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,829 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 09:16:33,830 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,830 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,831 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:33,832 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,832 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,833 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,834 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,835 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,835 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] is present: existingEntryTerm: 2 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,836 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,837 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:16:33,838 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,838 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 09:16:33,838 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,838 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,839 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,839 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,839 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,839 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,839 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:16:33,839 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,839 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,840 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,841 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,841 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,843 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:16:33,843 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,843 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:33,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,843 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 1 - updating term 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=2, votedFor=null 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,844 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:33,844 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,844 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,845 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:16:33,845 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:16:33,845 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,846 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testFlipMemberVotingStates, member-2-shard-people-oper_testFlipMemberVotingStates] 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,847 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:16:33,847 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,847 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,848 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 0, nextIndex: 1 09:16:33,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:33,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,850 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,850 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,850 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,851 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,851 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,852 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 0 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,853 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Setting last applied to 1 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,853 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,854 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 09:16:33,854 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:33,854 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 09:16:33,854 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:33,854 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 09:16:33,854 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:33,854 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 09:16:33,855 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:33,855 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:33,855 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,855 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,855 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 1, nextIndex: 2 09:16:33,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:33,856 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,856 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,857 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,859 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:16:33,859 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,859 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] is present: existingEntryTerm: 2 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 1 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=1, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 0 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from -1 to 0 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 1 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 0 to 1 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:16:33,860 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=1 09:16:33,864 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:16:33,864 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 09:16:33,864 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 09:16:33,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:16:33,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:16:33,866 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:16:33,866 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:16:33,867 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:33,867 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:33,867 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:16:33,867 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:16:33,868 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,868 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,868 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:33,869 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:33,869 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-oper_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:33,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-config_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:16:33,869 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:33,869 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-cars-config_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:16:33,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates)] 09:16:33,869 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates)] 09:16:33,869 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates)] 09:16:33,869 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,869 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:33,869 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: getLeaderAddress leaderId = member-1-shard-people-oper_testFlipMemberVotingStates peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:16:33,870 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates)] 09:16:33,878 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:33,879 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:33,879 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:33,879 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:33,880 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:33,880 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:33,880 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=42918366-9486-42da-9311-108be9b7271a}, logIndex: 2, isSendImmediate: true 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bf4c1abd-f994-4ad0-85d6-17d8b2c9a269}, logIndex: 2, isSendImmediate: true 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=170c1d9a-ee7a-4a4f-9bbd-cf0d54795d58}, logIndex: 2, isSendImmediate: true 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,881 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,881 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,882 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 09:16:33,881 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] 09:16:33,882 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testFlipMemberVotingStates=false, member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 09:16:33,882 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,882 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 09:16:33,882 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:33,882 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] 09:16:33,882 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:33,882 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 09:16:33,882 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,882 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=c687bf9d-9969-4d5e-b3a5-ca92e537d350}, logIndex: 2, isSendImmediate: true 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,883 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 09:16:33,883 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,884 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,885 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,886 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:33,886 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,886 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 3 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,886 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,887 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,887 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,888 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,888 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,888 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,888 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,888 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,889 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 3 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 3 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,891 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:33,891 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:16:33,891 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:16:33,892 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:33,892 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 3 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:33,892 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 0 for term 2 09:16:33,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:33,893 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,893 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:33,893 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,893 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:33,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:16:33,894 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] 09:16:33,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] 09:16:33,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] 09:16:33,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,894 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:33,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:16:33,894 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:33,894 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,894 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 09:16:33,895 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,894 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 2 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,895 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:33,895 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:33,895 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@13d8e0f2 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@13d8e0f2 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:33,896 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:33,896 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:33,896 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,896 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,896 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@149ff7d7 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@754cf471 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@149ff7d7 09:16:33,896 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@754cf471 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:33,897 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 2 09:16:33,897 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,897 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:33,898 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:16:33,898 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,898 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1098237329] 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50147a6 09:16:33,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1098237329] 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50147a6 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,898 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1233245020] 09:16:33,898 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:33,899 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1477169683] 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1477169683] 09:16:33,899 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1233245020] 09:16:33,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:16:33,899 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1629373211] 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1629373211] 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1132111738] 09:16:33,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:16:33,899 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1299530024] 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1299530024] 09:16:33,899 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1132111738] 09:16:33,899 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:16:33,900 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-928890318] 09:16:33,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-928890318] 09:16:33,900 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1393729446] 09:16:33,901 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:33,901 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2046509211] 09:16:33,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2046509211] 09:16:33,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1393729446] 09:16:33,902 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:16:33,902 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-454144234] 09:16:33,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-454144234] 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,905 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,905 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,906 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,906 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 09:16:33,906 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,906 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,907 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,907 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#928358097] 09:16:33,908 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:33,908 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1354733790] 09:16:33,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1354733790] 09:16:33,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#928358097] 09:16:33,908 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:16:33,908 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1422143422] 09:16:33,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1422143422] 09:16:33,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:33,909 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:33,909 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:33,909 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testFlipMemberVotingStates] 09:16:33,909 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,910 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-config_testFlipMemberVotingStates] 09:16:33,910 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 09:16:33,910 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,910 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 09:16:33,910 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 09:16:33,910 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: false 09:16:33,910 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates] 09:16:33,911 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-config_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 2 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 2 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 1 to 2 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:33,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 2 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 1 for term 2 09:16:33,912 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=2 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-people-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 09:16:33,912 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:33,913 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,913 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,913 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:33,913 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:33,913 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:33,914 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:16:33,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:33,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-config_testFlipMemberVotingStates 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:33,919 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,919 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-config_testFlipMemberVotingStates 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,919 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:33,919 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,919 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,919 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:16:33,919 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 09:16:33,919 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 09:16:33,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,920 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:33,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,920 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,920 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-people-oper_testFlipMemberVotingStates 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-1-shard-people-config_testFlipMemberVotingStates, member-2-shard-people-config_testFlipMemberVotingStates] 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,921 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,921 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,921 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,921 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:33,921 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 09:16:33,922 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 09:16:33,922 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,922 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-1-shard-people-oper_testFlipMemberVotingStates] 09:16:33,922 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,922 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 09:16:33,922 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,922 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,922 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:33,923 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,923 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:33,923 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 09:16:33,923 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,923 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,923 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 09:16:33,923 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,923 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:33,924 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:16:33,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,924 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 09:16:33,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,925 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,925 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,925 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,925 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:16:33,926 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,926 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,927 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 09:16:33,927 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 09:16:33,928 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:16:33,930 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 2, lastIndex: 2, isVoting: true 09:16:33,930 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:33,931 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 09:16:33,931 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,931 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 09:16:33,931 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,931 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:33,931 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:33,932 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:33,931 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,932 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,932 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:33,932 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,932 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [] 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,936 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,936 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:33,936 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:33,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Received ElectionTimeout 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-1-shard-cars-oper_testFlipMemberVotingStates] 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:33,938 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:33,938 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:33,938 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 09:16:33,938 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:33,945 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 2 - updating term 09:16:33,945 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,946 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:33,946 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:33,946 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:16:33,948 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:33,948 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 2 - switching to Follower 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=3, votedFor=null 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:33,950 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:33,950 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 09:16:33,960 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,10 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,112 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,213 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,241 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 09:16:34,241 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 344.9 ms 09:16:34,241 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 09:16:34,241 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 09:16:34,241 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 345.0 ms 09:16:34,242 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 09:16:34,242 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 09:16:34,242 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 09:16:34,242 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 09:16:34,242 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,242 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,242 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 346.4 ms 09:16:34,242 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,242 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 09:16:34,242 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:34,242 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:34,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:34,245 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:34,245 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:34,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:34,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:34,246 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:34,248 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 09:16:34,249 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 09:16:34,249 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 357.1 ms 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer succeeded after change to non-voting 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,255 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:34,256 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 2, nextIndex: 3 09:16:34,256 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:16:34,258 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:16:34,263 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,264 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] 09:16:34,264 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:34,264 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:34,264 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:34,264 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:16:34,264 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 09:16:34,265 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 09:16:34,265 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:16:34,265 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:16:34,265 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:34,265 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:16:34,265 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:16:34,266 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:16:34,266 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:34,266 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:16:34,267 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:34,267 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:34,267 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:34,267 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:34,267 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]] 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,268 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a3bc58a9-b955-41b1-8422-aa8b384c64ce}, logIndex: 3, isSendImmediate: true 09:16:34,268 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,268 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]] 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,268 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=b543c599-5e86-4350-9c88-57a2d44f5f0a}, logIndex: 3, isSendImmediate: true 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]], state: Idle 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-config_testFlipMemberVotingStates 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]] 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,269 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=10fb7334-30df-4361-a0d5-a29feceef7cc}, logIndex: 3, isSendImmediate: true 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]], state: Idle 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-oper_testFlipMemberVotingStates 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]] 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=b945c8df-8f2a-455a-b2f9-75baf7297df3}, logIndex: 3, isSendImmediate: true 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:34,270 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] 09:16:34,270 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,270 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:16:34,270 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:16:34,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:16:34,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:16:34,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:16:34,271 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:16:34,271 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:16:34,271 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,271 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,272 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,272 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:34,273 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:34,273 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,273 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,273 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,273 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,274 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,274 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,274 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,274 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,274 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,275 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,275 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,275 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,275 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,275 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,275 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,275 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,275 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,276 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,276 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,276 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:34,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2329cada 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2329cada 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Setting last applied to 3 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,277 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,277 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-config_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:34,278 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c647452 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c647452 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-config_testFlipMemberVotingStates 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-config_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,278 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Setting last applied to 3 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] has been successfully replicated to a majority of followers 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - Leader changed to non-voting - trying leadership transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a26fe80 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b1c105a 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a26fe80 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b1c105a 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:34,279 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,279 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,280 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,280 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-oper_testFlipMemberVotingStates 09:16:34,280 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-cars-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Sending AppendEntries to follower member-1-shard-people-oper_testFlipMemberVotingStates: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,280 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,280 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,281 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,282 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,283 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 09:16:34,283 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,283 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 09:16:34,284 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,284 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,285 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,285 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,285 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates, votingState=VOTING]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,286 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,286 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,287 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStates, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates, votingState=NON_VOTING]] 09:16:34,287 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,287 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,287 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-people-config_testFlipMemberVotingStates] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-cars-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates] 09:16:34,288 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,288 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]]]] 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Number of entries to be appended = 1 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]]] is present: existingEntryTerm: 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): After cleanup, lastIndex: 3, entries to be added from: 1 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Log size is now 2 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Commit index set to 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-1-shard-people-oper_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Setting last applied to 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Moving last applied index from 2 to 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Applying state for log index 3 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]] 09:16:34,289 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: fakeSnapshot purging log to 2 for term 3 09:16:34,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Persisting ApplyJournalEntries with index=3 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,290 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:34,290 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,290 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,290 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,290 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,290 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 09:16:34,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:34,291 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-oper_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 09:16:34,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:34,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:34,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:34,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 09:16:34,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:34,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,292 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,292 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:34,292 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStates] 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-oper_testFlipMemberVotingStates 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,296 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:34,296 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStates] 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:34,297 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStates] 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-people-config_testFlipMemberVotingStates 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:34,297 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:34,297 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:34,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,298 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:16:34,300 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,300 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,300 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,300 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStates 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,301 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStates 09:16:34,301 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testFlipMemberVotingStates, member-3-shard-people-oper_testFlipMemberVotingStates] 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStates, member-3-shard-cars-oper_testFlipMemberVotingStates] 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:34,304 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:16:34,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:16:34,304 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStates, member-3-shard-people-config_testFlipMemberVotingStates] 09:16:34,305 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:34,305 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,305 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 09:16:34,305 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,305 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 09:16:34,305 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:34,305 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,305 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 09:16:34,305 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testFlipMemberVotingStates] 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,306 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:34,306 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:34,306 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,306 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStates is no longer the leader 09:16:34,308 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,308 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Cluster state: [] 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than follower's term 3 - updating term 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 3, lastTerm: 3 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,309 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:16:34,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStates is no longer the leader 09:16:34,309 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 09:16:34,309 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:16:34,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStates is no longer the leader 09:16:34,309 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,309 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 09:16:34,310 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStates, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,310 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:16:34,310 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:16:34,310 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStates, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,310 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStates, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,310 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:16:34,310 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-1-shard-cars-config_testFlipMemberVotingStates] 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:16:34,311 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:34,311 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,312 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,312 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,312 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:34,312 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:34,312 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:16:34,312 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:34,313 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries [leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than leader's term 3 - switching to Follower 09:16:34,313 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=4, votedFor=null 09:16:34,313 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:16:34,313 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,313 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:34,313 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:34,313 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,314 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 09:16:34,315 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,316 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:34,316 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:34,316 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:16:34,316 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 09:16:34,316 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,316 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,317 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:16:34,318 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 09:16:34,328 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,328 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:34,328 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:16:34,328 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] 09:16:34,328 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#954170170] 09:16:34,329 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] 09:16:34,329 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:34,329 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:34,329 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#563568057] 09:16:34,329 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1487095816] 09:16:34,329 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:16:34,330 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1098237329] 09:16:34,330 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1098237329] 09:16:34,330 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1233245020] 09:16:34,330 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:34,330 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1477169683] 09:16:34,330 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1477169683] 09:16:34,331 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1233245020] 09:16:34,331 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:16:34,331 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1629373211] 09:16:34,331 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1629373211] 09:16:34,331 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1132111738] 09:16:34,331 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:16:34,331 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1299530024] 09:16:34,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#-1299530024] 09:16:34,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1132111738] 09:16:34,332 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:16:34,332 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-928890318] 09:16:34,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-928890318] 09:16:34,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1393729446] 09:16:34,332 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:34,332 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2046509211] 09:16:34,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2046509211] 09:16:34,333 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1393729446] 09:16:34,333 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:16:34,333 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-454144234] 09:16:34,333 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-454144234] 09:16:34,333 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#928358097] 09:16:34,333 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:34,333 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1354733790] 09:16:34,333 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1354733790] 09:16:34,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#928358097] 09:16:34,334 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:16:34,334 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1422143422] 09:16:34,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1422143422] 09:16:34,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1776618357] 09:16:34,334 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:34,334 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,334 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-188993998] 09:16:34,335 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:34,335 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 09:16:34,336 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 09:16:34,336 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:16:34,336 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onShutDown 09:16:34,336 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onShutDown 09:16:34,337 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStates 09:16:34,337 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStates 09:16:34,338 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:16:34,338 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:16:34,338 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:34,338 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:34,339 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 09:16:34,339 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:16:34,339 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onShutDown 09:16:34,339 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onShutDown 09:16:34,339 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStates 09:16:34,339 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStates 09:16:34,340 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:16:34,340 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:16:34,344 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:34,344 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:34,353 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:34,353 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:34,353 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:34,354 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:34,356 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:16:34,376 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:34,376 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,377 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 09:16:34,377 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:16:34,377 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onShutDown 09:16:34,377 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:34,378 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onShutDown 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48444201 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48444201 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,378 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,378 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,378 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStates 09:16:34,382 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:34,382 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:34,383 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testFlipMemberVotingStates, matchIndex: 3, lastIndex: 3, isVoting: true 09:16:34,384 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Follower's log matches - sending ElectionTimeout 09:16:34,384 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,384 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,384 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leader transfer complete 09:16:34,384 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Received TimeoutNow - switching to Candidate 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] to peer member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,386 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:34,387 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:16:34,387 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,387 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Cluster state: [] 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] with higher term appears reachable, updating term. 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=null 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer in progress - processing RequestVote 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 3, lastTerm: 3 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Set currentTerm=5, votedFor=member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:16:34,388 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStates is no longer the leader 09:16:34,388 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:16:34,388 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=-1] 09:16:34,388 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 09:16:34,389 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:16:34,389 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testFlipMemberVotingStates, member-2-shard-cars-config_testFlipMemberVotingStates] 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:16:34,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:16:34,390 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:34,391 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 09:16:34,391 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 13.91 ms 09:16:34,391 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer succeeded - sending PoisonPill 09:16:34,391 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStates, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderPayloadVersion=13] 09:16:34,392 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:16:34,392 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:16:34,392 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStates 09:16:34,392 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:16:34,392 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:16:34,393 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStates updated: matchIndex: 3, nextIndex: 4 09:16:34,393 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:34,393 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:34,393 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:34,393 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 09:16:34,393 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:16:34,393 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onShutDown 09:16:34,393 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onShutDown 09:16:34,394 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStates 09:16:34,394 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStates 09:16:34,394 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:16:34,394 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:16:34,395 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:34,395 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:34,656 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:16:35,366 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [2064743092203294317] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:35,416 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-2124824066484412623] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:35,420 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:35,420 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:35,422 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:16:35.425] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:35,437 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:16:35,438 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 09:16:35,439 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 09:16:35,439 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:16:35,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onShutDown 09:16:35,439 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:35,439 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onShutDown 09:16:35,439 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Initiating leader transfer 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@498cb1da 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@498cb1da 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:35,440 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:35,446 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:35,446 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:35,446 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b9a18c5 09:16:35,446 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b9a18c5 09:16:35,446 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:35,446 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:35,447 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:35,447 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:36,455 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 09:16:36,455 AM [cluster-test-shard-dispatcher-35] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.016 s 09:16:36,455 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 09:16:36,456 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 09:16:36,456 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 09:16:36,457 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStates 09:16:36,545 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 09:16:36,545 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.106 s 09:16:36,545 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 09:16:36,545 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 09:16:36,545 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Leadership transfer expired 09:16:36,546 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStates 09:16:36,547 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:16:36,547 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:16:36,548 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:16:36,549 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:36,549 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 09:16:36,549 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onShutDown 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:36,550 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55de7c6 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@55de7c6 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:36,550 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onShutDown 09:16:36,550 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Initiating leader transfer 09:16:36,550 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStates, leaderId=null, leaderPayloadVersion=13] 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a57eac8 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a57eac8 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Attempting to transfer leadership 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:36,551 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStates: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:16:37,655 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 09:16:37,655 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 09:16:37,655 AM [cluster-test-shard-dispatcher-35] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.105 s 09:16:37,655 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 09:16:37,655 AM [cluster-test-shard-dispatcher-10] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.104 s 09:16:37,655 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 09:16:37,655 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 09:16:37,655 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer failed - sending PoisonPill 09:16:37,655 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 09:16:37,655 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Leadership transfer expired 09:16:37,656 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStates 09:16:37,656 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStates 09:16:37,657 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:16:37,657 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:16:37,661 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:37,661 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:37,668 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:16:37.670] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:37,710 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:37,724 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [2076565985944595153] 09:16:37,725 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:16:37,730 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:37,730 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:16:37,732 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:37,732 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:37,748 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,751 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,752 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:16:37,752 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:37,752 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:16:37,755 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:16:37,760 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:16:37,761 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,767 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:16:37,767 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,767 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,768 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:37,768 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,768 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,771 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:37,773 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@16c98373} 09:16:37,773 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:16:37,774 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:37,774 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,775 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,775 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,776 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:37,776 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,776 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testModuleShardLeaderMovement, persistent : true 09:16:37,777 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,777 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1632952207 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,777 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#91610920] 09:16:37,777 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1632952207] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,777 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 09:16:37,778 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testModuleShardLeaderMovement, persistent : true 09:16:37,778 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testModuleShardLeaderMovement with journal batch size 1 09:16:37,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-49555988] 09:16:37,779 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-1118997094 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,780 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,780 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,780 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#583997558] 09:16:37,781 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:37,781 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:37,781 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:16:37,781 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 09:16:37,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-config_testModuleShardLeaderMovement 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,782 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testModuleShardLeaderMovement is no longer the leader 09:16:37,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:37,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 09:16:37,783 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#91610920] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 09:16:37,783 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#583997558] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,783 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:37,783 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-49555988] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:37,783 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-1118997094] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:37,783 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:37,784 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:16:37,784 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderId=member-1-shard-people-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:37,784 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:37,785 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:16:37,785 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:37,785 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 09:16:37,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:37,786 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:16:37,786 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:37,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:37,786 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:37,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 09:16:37,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 09:16:37,787 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:37,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:37,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:37,788 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:37,788 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:16:37,788 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:37,788 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:16:37,788 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:37,824 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] 09:16:37,825 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:37,825 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:37,825 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,828 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,832 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:16:37,835 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-894118153] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,837 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,838 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:37,839 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@2b95d80f} 09:16:37,839 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:16:37,840 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:37,840 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801] 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#816169548] 09:16:37,841 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-oper_testModuleShardLeaderMovement, persistent : true 09:16:37,842 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 09:16:37,842 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-508649700 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,842 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-2013388606 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,842 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#-490365375] 09:16:37,842 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,842 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#816169548] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,843 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/shard-member-1:snapshot-read#724877613] 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:37,843 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testModuleShardLeaderMovement with journal batch size 1 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:37,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:16:37,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,843 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:37,843 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:16:37,843 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:37,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:37,844 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:37,844 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:16:37,844 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [] 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:37,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:16:37,845 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:37,845 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testModuleShardLeaderMovement is no longer the leader 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:37,845 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:37,845 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 09:16:37,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Received ElectionTimeout 09:16:37,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [] 09:16:37,846 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:16:37,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:37,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:37,846 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:37,846 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:16:37,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-people-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:37,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 09:16:37,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:37,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-894118153] 09:16:37,894 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:16:37,894 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801] 09:16:37,912 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:16:37,927 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-187643359609441150] 09:16:37,929 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:16:37,932 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:37,932 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:16:37,936 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:37,936 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:37,946 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,949 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,955 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:16:37,964 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,964 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:16:37,966 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:16:37,968 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@16cfe622} 09:16:37,969 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:16:37,970 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,971 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:37,976 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:37,978 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:16:37,980 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:16:37,980 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1768062928] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:37,981 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:16:37,990 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:37,991 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@18080a8b} 09:16:37,991 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:16:37,995 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:38,13 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:38,33 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4046616139389026374] 09:16:38,35 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:16:38,37 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:38,37 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:16:38,39 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:38,39 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:38,60 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:38,62 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:38,65 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:16:38,82 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:16:38,84 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:16:38,87 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:38,102 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:38,102 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:38,102 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,102 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:38,102 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,106 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:16:38,110 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:38,117 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:16:38,118 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:38,118 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,118 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:38,119 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@34abc503} 09:16:38,119 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:16:38,121 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:38,121 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:38,125 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:38,127 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:16:38,128 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1570293945] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:38,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:16:38,131 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:16:38,131 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:38,131 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,132 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:38,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3a6c2de6} 09:16:38,133 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:16:38,134 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:38,749 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:16:38,749 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,750 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,750 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:16:38,750 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:38,751 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,760 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,760 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,760 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,760 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,760 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,973 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:38,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,973 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:38,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,973 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:38,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:38,973 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,39 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:16:39,39 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:39,40 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,41 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 09:16:39,45 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,45 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521, primaryVersion=13] 09:16:39,49 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,50 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-2-shard-cars-config_testModuleShardLeaderMovement, persistent : true 09:16:39,50 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,50 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-166856056 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,51 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-166856056] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,51 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-149446466] 09:16:39,51 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/shard-member-2:snapshot-read#-149446466] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,52 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 09:16:39,52 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 09:16:39,52 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 09:16:39,53 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:39,53 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,53 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 09:16:39,53 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:39,53 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,53 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:39,53 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:39,53 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:16:39,53 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 09:16:39,53 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,53 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,54 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:16:39,54 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 09:16:39,54 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@31e210c5, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=null] 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testModuleShardLeaderMovement: 229 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,55 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,55 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,55 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 09:16:39,56 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:39,56 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,56 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:39,56 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:39,56 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,57 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@34993d48, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 09:16:39,57 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1762800766] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 09:16:39,58 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1762800766] 09:16:39,58 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 502.0 μs 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 09:16:39,58 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,58 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 88.20 μs 09:16:39,59 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,59 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,59 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1762800766] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,59 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 09:16:39,60 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=2cff74ae-e347-4460-b62c-9f8f51a67c1f}, logIndex: 0, isSendImmediate: true 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,61 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 09:16:39,62 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING]] 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 09:16:39,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521 09:16:39,64 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:16:39,65 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,65 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 09:16:39,65 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,65 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:39,65 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,65 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:16:39,65 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,65 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:16:39,66 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,66 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,66 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:39,66 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,66 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:16:39,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 0 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:39,67 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 09:16:39,67 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1768062928] 09:16:39,68 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:16:39,68 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 09:16:39,68 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 0 09:16:39,68 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:16:39,68 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 0 09:16:39,68 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,68 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,69 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 09:16:39,69 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,69 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:16:39,69 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:39,69 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,69 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 09:16:39,71 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,71 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801, primaryVersion=13] 09:16:39,73 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,73 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 09:16:39,73 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1126185585] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,74 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-824205062 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,74 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1930959883] 09:16:39,74 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-824205062] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 1 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7e18f2b2, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=null] 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 09:16:39,75 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: 229 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,76 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 09:16:39,79 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 09:16:39,79 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/shard-member-2:snapshot-read#1930959883] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:39,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:16:39,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:39,80 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:16:39,81 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:39,81 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,81 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@65b25193, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:16:39,81 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:39,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:16:39,82 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:16:39,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to -1 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 09:16:39,82 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1103635085] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 318.0 μs 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,82 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1103635085] 09:16:39,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:16:39,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 69.96 μs 09:16:39,83 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1103635085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement] 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 09:16:39,84 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4d977bc4-6d87-4770-9498-4e9471e704c4}, logIndex: 0, isSendImmediate: true 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 09:16:39,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:39,87 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801 09:16:39,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:16:39,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,88 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 0, nextIndex: 1 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 0 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:39,88 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:39,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1768062928] 09:16:39,88 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 09:16:39,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:16:39,88 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1126185585] 09:16:39,89 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 09:16:39,89 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1126185585] 09:16:39,89 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:16:39,89 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:39,90 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,90 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 0 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 0 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=0 09:16:39,90 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:16:39,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521, primaryVersion=13] 09:16:39,94 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,95 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testModuleShardLeaderMovement, persistent : true 09:16:39,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,95 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#922441559] 09:16:39,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#665563089 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#665563089] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,96 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/shard-member-3:snapshot-read#922441559] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,96 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testModuleShardLeaderMovement with journal batch size 1 09:16:39,96 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true], state: Idle 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 09:16:39,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,97 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 09:16:39,97 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:16:39,97 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:39,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,97 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 09:16:39,97 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:16:39,97 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2703e6cd, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 09:16:39,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testModuleShardLeaderMovement: 229 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement), Chunk: 1/1 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,98 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 09:16:39,99 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@11387669, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 09:16:39,100 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:39,100 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2020792068] 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2020792068] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 09:16:39,101 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:16:39,101 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#2020792068] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,101 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 65.18 μs 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 938.8 μs 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a journal response 09:16:39,101 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]] 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,102 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4434b5c3-62b5-4d7d-b4d6-55e593206ae1}, logIndex: 1, isSendImmediate: true 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,103 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement, votingMember=true] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 2 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,104 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Log size is now 1 09:16:39,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,105 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521 09:16:39,105 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 09:16:39,105 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,105 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,105 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,106 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:16:39,106 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,106 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,106 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,106 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,106 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,106 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:16:39,106 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,107 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1570293945] 09:16:39,107 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:39,107 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 09:16:39,107 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:16:39,107 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Setting last applied to 1 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 09:16:39,107 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,107 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:39,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,108 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801, primaryVersion=13] 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,109 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Commit index set to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Setting last applied to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 09:16:39,109 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,111 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,111 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-oper_testModuleShardLeaderMovement, persistent : true 09:16:39,112 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testModuleShardLeaderMovement with journal batch size 1 09:16:39,112 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#351737831] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,112 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1501586865 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,112 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-1669345096] 09:16:39,113 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1501586865] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,113 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/shard-member-3:snapshot-read#-1669345096] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true], state: Idle 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: lastSequenceNumber prior to capture: 3 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating capture snapshot for new server member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: CaptureSnapshotReply received by actor 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2cd40e76, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testModuleShardLeaderMovement, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:16:39,114 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendInstallSnapshot 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: 229 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testModuleShardLeaderMovement, member-3-shard-cars-oper_testModuleShardLeaderMovement), Chunk: 1/1 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 3 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,115 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 09:16:39,115 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:39,115 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:39,115 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:16:39,115 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:16:39,115 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,115 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:39,116 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:16:39,116 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:16:39,116 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 09:16:39,117 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]" message is greater than follower's term 0 - updating term 09:16:39,117 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,117 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,117 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:16:39,117 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,117 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=-1] 09:16:39,117 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@312f002e, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Snapshot success - sequence number: 1 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from -1 to 0 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Set currentTerm=1, votedFor=null 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-839891888] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 09:16:39,118 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-839891888] 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 387.5 μs 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 09:16:39,118 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,118 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:16:39,119 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: SnapshotComplete received 09:16:39,119 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 86.04 μs 09:16:39,119 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-839891888] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:39,119 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a journal response 09:16:39,119 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: handled a snapshot response 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, chunkIndex=1, success=true] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: New server configuration : [ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=da8f756b-d967-465a-9f32-9705c5f814d9}, logIndex: 1, isSendImmediate: true 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,120 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testModuleShardLeaderMovement: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,121 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, newServerAddress=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement, votingMember=true] 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,122 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]]]] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Number of entries to be appended = 1 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 1 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,122 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testModuleShardLeaderMovement, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testModuleShardLeaderMovement, address=null, votingState=VOTING]] 09:16:39,123 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-959253801 09:16:39,123 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Log size is now 2 09:16:39,123 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 09:16:39,123 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:39,123 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,123 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 09:16:39,124 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:16:39,124 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,124 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:16:39,124 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testModuleShardLeaderMovement is no longer the leader 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:39,124 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:16:39,124 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]]] 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 09:16:39,124 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:16:39,124 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1570293945] 09:16:39,124 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:16:39,124 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#351737831] 09:16:39,124 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#351737831] 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Setting last applied to 1 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,125 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:39,125 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,126 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:16:39,127 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:39,127 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,127 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,126 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Commit index set to 1 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Setting last applied to 1 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Moving last applied index from 0 to 1 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]] 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: fakeSnapshot purging log to 0 for term 1 09:16:39,127 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Persisting ApplyJournalEntries with index=1 09:16:39,127 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,127 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,127 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,128 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,130 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard cars, datastoreType Config 09:16:39,130 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] 09:16:39,130 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:39,130 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,131 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,132 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:16:39,132 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:16:39,133 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] 09:16:39,133 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:39,133 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,133 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] 09:16:39,136 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:39,136 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,137 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,137 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,137 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard cars, datastoreType Config 09:16:39,137 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] 09:16:39,137 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:39,137 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,137 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,137 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:16:39,137 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-1-shard-cars-config_testModuleShardLeaderMovement peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,143 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-2-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[pekko://cluster-test@127.0.0.1:2559/temp/member-2-shard-cars-config_testModuleShardLeaderMovement$e]] 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66fc306 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66fc306 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,145 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,147 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,147 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,147 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,147 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,147 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 09:16:39,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,151 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:39,151 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,151 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:16:39,151 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,151 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 09:16:39,151 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,151 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:16:39,151 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,152 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,152 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,152 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 1 - switching to Follower 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=null 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=2, votedFor=member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,152 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,152 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:16:39,153 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,153 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:39,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:39,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,153 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:39,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:16:39,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,154 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,154 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 9.727 ms 09:16:39,154 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,154 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,154 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,155 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,155 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,155 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,155 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,160 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,160 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,160 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,160 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:16:39,161 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1770809180] 09:16:39,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:39,161 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,161 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#331485705] 09:16:39,162 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#2146648975] 09:16:39,162 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:39,162 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-1842688521] 09:16:39,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,162 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,162 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,163 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,163 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard cars, datastoreType Config 09:16:39,163 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,163 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,163 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,163 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,163 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:16:39,163 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: getLeaderAddress leaderId = member-2-shard-cars-config_testModuleShardLeaderMovement peerAddress = pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onRequestLeadership RequestLeadership [requestedFollowerId=member-3-shard-cars-config_testModuleShardLeaderMovement, replyTo=Actor[pekko://cluster-test@127.0.0.1:2557/temp/member-3-shard-cars-config_testModuleShardLeaderMovement$e]] 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@756ca980 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@756ca980 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,165 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,166 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,166 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 09:16:39,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,170 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 2 - switching to Follower 09:16:39,170 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 2 - updating term 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=null 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 3, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=3, votedFor=member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,171 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,171 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,172 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-1-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:39,172 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:16:39,173 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,173 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,173 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,173 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 8.464 ms 09:16:39,173 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Leadership transferred successfully to member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,173 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,174 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,174 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,174 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:16:39,174 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,174 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-1755057577] 09:16:39,174 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,174 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:39,174 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#1981628472] 09:16:39,175 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,175 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onShutDown 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 09:16:39,176 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d90582f 09:16:39,176 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d90582f 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-config_testModuleShardLeaderMovement, matchIndex: 1, lastIndex: 1, isVoting: true 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Follower's log matches - sending ElectionTimeout 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-1-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leader transfer complete 09:16:39,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:16:39,178 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,178 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,178 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,178 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:39,179 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Received TimeoutNow - switching to Candidate 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] to peer member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,179 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,180 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:16:39,180 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,180 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Cluster state: [] 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than follower's term 3 - updating term 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testModuleShardLeaderMovement, member-2-shard-cars-config_testModuleShardLeaderMovement] 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Cluster state: [] 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] with higher term appears reachable, updating term. 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1]" message is greater than leader's term 3 - switching to Follower 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=null 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer in progress - processing RequestVote 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 1 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Set currentTerm=4, votedFor=member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:16:39,182 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,182 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:16:39,182 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:16:39,183 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,183 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:16:39,183 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testModuleShardLeaderMovement is no longer the leader 09:16:39,183 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:16:39,183 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=-1] 09:16:39,183 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:16:39,184 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:39,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,184 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 7.725 ms 09:16:39,184 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer succeeded - sending PoisonPill 09:16:39,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,184 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,185 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testModuleShardLeaderMovement 09:16:39,185 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:16:39,185 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:16:39,186 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:16:39,186 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testModuleShardLeaderMovement, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderPayloadVersion=13] 09:16:39,186 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:16:39,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,187 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:39,187 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testModuleShardLeaderMovement updated: matchIndex: 1, nextIndex: 2 09:16:39,187 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 09:16:39,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:39,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 09:16:39,188 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,188 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:16:39,188 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:16:39,188 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:16:39,189 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:39,190 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:39,195 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:39,195 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:39,196 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:39,196 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:39,197 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:16:39,227 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:39,227 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,228 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:16:39,228 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onShutDown 09:16:39,229 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testModuleShardLeaderMovement 09:16:39,230 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:16:39,230 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:16:39,230 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:39,232 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 09:16:39,232 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 09:16:39,233 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testModuleShardLeaderMovement 09:16:39,233 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:16:39,233 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:16:39,235 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:39,235 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:39,768 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:16:40,208 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-4046616139389026374] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:40,254 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:40,255 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:40,257 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-187643359609441150] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:40,262 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:16:40.264] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:40,283 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:16:40,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:40,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 09:16:40,284 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onShutDown 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating leader transfer 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d899a69 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d899a69 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:40,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onShutDown 09:16:40,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@78597077 09:16:40,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@78597077 09:16:40,286 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:40,286 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testModuleShardLeaderMovement 09:16:44,869 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:16:44,870 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:16:44,871 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:44,871 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:44,871 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:44,871 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:45,878 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:46,757 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:47,825 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:16:49,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 09:16:49,418 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.134 s 09:16:49,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 09:16:49,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 09:16:49,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Leadership transfer expired 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [WARN] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testModuleShardLeaderMovement, member-3-shard-cars-config_testModuleShardLeaderMovement] 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 09:16:49,419 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:49,420 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 09:16:49,420 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:16:49,420 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testModuleShardLeaderMovement 09:16:49,421 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:16:49,421 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:16:49,422 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:16:49,422 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:49,423 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:49,423 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onShutDown 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating leader transfer 09:16:49,423 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testModuleShardLeaderMovement, leaderId=null, leaderPayloadVersion=13] 09:16:49,423 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onShutDown 09:16:49,423 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5af1fbb8 09:16:49,423 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5af1fbb8 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e75b27a 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@e75b27a 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Attempting to transfer leadership 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:49,423 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:49,424 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testModuleShardLeaderMovement 09:16:53,147 AM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:16:53,147 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testModuleShardLeaderMovement, member-2-shard-cars-oper_testModuleShardLeaderMovement] 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testModuleShardLeaderMovement: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.725 s 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer failed - sending PoisonPill 09:16:53,148 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:16:53,148 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 09:16:53,148 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:16:53,149 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 09:16:53,149 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:16:53,149 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:16:53,151 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:53,151 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:53,154 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:16:53.155] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:53,179 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:53,193 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8542156616190167047] 09:16:53,193 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:16:53,196 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:53,196 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:16:53,197 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:53,197 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:53,208 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,210 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:16:53,210 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:16:53,211 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:16:53,213 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,217 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:16:53,224 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#986661891] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,226 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,229 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,235 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:53,241 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@73f7b81d} 09:16:53,241 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:16:53,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 09:16:53,242 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,244 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,247 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:16:53,248 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:16:53,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 09:16:53,249 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,249 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,249 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#107517298] 09:16:53,249 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardReplica, persistent : true 09:16:53,249 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,250 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardReplica with journal batch size 1 09:16:53,250 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#1871712101] 09:16:53,250 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#107517298] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,250 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#2063970902 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 09:16:53,251 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:16:53,250 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardReplica, persistent : true 09:16:53,251 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardReplica with journal batch size 1 09:16:53,252 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1883457575 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 09:16:53,252 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#7866791] 09:16:53,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,252 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,253 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:16:53,253 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 09:16:53,253 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,253 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,253 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:16:53,253 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 09:16:53,254 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:16:53,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 09:16:53,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 09:16:53,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,256 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 09:16:53,256 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,256 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,257 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 09:16:53,257 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,258 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 09:16:53,258 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/shard-member-1:snapshot-read#1871712101] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,258 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#2063970902] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,258 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1790376251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,258 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1883457575] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,258 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/shard-member-1:snapshot-read#7866791] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,259 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:16:53,260 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@5c9b6469} 09:16:53,260 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:16:53,262 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,262 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-426916030] 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,263 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#10486462] 09:16:53,263 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardReplica, persistent : true 09:16:53,264 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardReplica, persistent : true 09:16:53,264 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardReplica with journal batch size 1 09:16:53,264 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardReplica with journal batch size 1 09:16:53,266 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/shard-member-1:snapshot-read#-1394198643] 09:16:53,267 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#626915391 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 09:16:53,267 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#2084892524 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,267 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/shard-member-1:snapshot-read#753744084] 09:16:53,267 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#-426916030] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,267 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#10486462] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,267 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,268 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,268 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:16:53,268 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 09:16:53,268 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,268 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 09:16:53,268 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,268 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,268 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 09:16:53,269 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,269 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,269 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:16:53,269 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 09:16:53,269 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,269 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 09:16:53,269 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,269 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,269 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 09:16:53,273 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:53,285 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5779974178252173603] 09:16:53,286 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:16:53,293 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:53,293 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:16:53,294 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:53,294 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:53,324 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,325 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,328 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:16:53,345 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1790959106] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,347 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:16:53,356 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:16:53,356 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 09:16:53,357 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 09:16:53,360 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:53,360 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@5c2c6f6f} 09:16:53,361 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:16:53,361 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,361 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 09:16:53,361 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,363 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,365 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:16:53,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 09:16:53,373 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 09:16:53,374 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 09:16:53,374 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 09:16:53,375 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 09:16:53,376 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1282706523] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#1038817736] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-569616326] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1919536955] 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-489179751 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1438062251 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#74797967] 09:16:53,376 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-489179751] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,378 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,378 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,378 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:16:53,378 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:16:53,383 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 09:16:53,383 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,383 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,387 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:16:53,390 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:16:53,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 09:16:53,391 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,391 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 09:16:53,391 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,391 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,391 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 09:16:53,392 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:16:53,393 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-1919536955] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,393 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1438062251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,393 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#74797967] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,393 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:16:53,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@375e8ec6} 09:16:53,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 09:16:53,397 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 09:16:53,397 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 09:16:53,397 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 09:16:53,398 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 09:16:53,398 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-1625865469] 09:16:53,398 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-1820976857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,399 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#1589063916] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,399 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-925626690 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 09:16:53,399 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 09:16:53,400 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1186231068 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 09:16:53,400 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#2100443313] 09:16:53,400 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,400 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,401 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:16:53,401 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 09:16:53,401 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,401 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 09:16:53,401 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,401 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,401 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 09:16:53,401 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 09:16:53,401 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 09:16:53,402 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,402 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 09:16:53,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:16:53,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,404 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,405 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,405 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:53,405 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 09:16:53,416 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:53,427 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3435325109717566836] 09:16:53,428 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:16:53,430 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:16:53,430 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:16:53,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:53,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:53,442 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,446 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,448 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:16:53,460 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#724955445] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,460 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:16:53,465 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:53,466 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:16:53,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 09:16:53,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 09:16:53,471 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@485a3d25} 09:16:53,471 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:16:53,471 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,472 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,472 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 09:16:53,473 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:53,475 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:16:53,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 09:16:53,479 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardReplica, persistent : true 09:16:53,479 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,480 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-2064801012] 09:16:53,480 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-949031183] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,480 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/shard-member-3:snapshot-read#-2064801012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,480 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-949031183 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 09:16:53,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-354846803] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,481 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:16:53,483 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardReplica, persistent : true 09:16:53,483 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardReplica with journal batch size 1 09:16:53,483 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#694837571] 09:16:53,484 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#308246530 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 09:16:53,484 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1406736398] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,484 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:16:53,484 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/shard-member-3:snapshot-read#694837571] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,486 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,491 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#308246530] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,492 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:16:53,491 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardReplica is no longer the leader 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:16:53,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,493 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,493 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardReplica with journal batch size 1 09:16:53,493 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,494 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:16:53,494 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,494 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,495 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,495 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardReplica is no longer the leader 09:16:53,495 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:53,495 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:16:53,495 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7a0d065} 09:16:53,495 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:16:53,496 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:53,496 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:53,496 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 09:16:53,497 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,497 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:53,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#660775508] 09:16:53,498 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 09:16:53,498 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardReplica, persistent : true 09:16:53,498 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 09:16:53,498 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:16:53,498 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 09:16:53,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#660775508] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:53,498 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1475900362 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 09:16:53,499 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardReplica with journal batch size 1 09:16:53,499 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-428170809] 09:16:53,499 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1475900362] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:53,499 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 09:16:53,499 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardReplica, persistent : true 09:16:53,499 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,499 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,500 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,500 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardReplica is no longer the leader 09:16:53,500 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,500 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,500 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/shard-member-3:snapshot-read#-829325610] 09:16:53,500 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-118599005 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 09:16:53,500 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardReplica with journal batch size 1 09:16:53,500 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:53,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#2101111221] 09:16:53,501 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:16:53,501 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 09:16:53,501 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,501 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:53,501 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:16:53,501 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardReplica is no longer the leader 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:53,501 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:16:53,501 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:53,501 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,583 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardReplica is no longer the leader 09:16:53,584 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 09:16:53,584 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,603 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardReplica is no longer the leader 09:16:53,604 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 09:16:53,604 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 09:16:53,613 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:53,613 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 09:16:53,613 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 09:16:53,613 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:53,613 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 09:16:53,614 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:53,614 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,614 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardReplica is no longer the leader 09:16:53,614 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 09:16:53,614 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 09:16:53,673 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:16:53,673 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower): Received ElectionTimeout - switching to Candidate 09:16:53,673 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 09:16:53,673 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:53,673 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 09:16:53,674 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:16:53,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:53,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardReplica is no longer the leader 09:16:53,674 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 09:16:53,674 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 09:16:53,973 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:53,973 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:53,973 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:53,973 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:53,973 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 09:16:53,973 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 09:16:54,3 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,3 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:54,3 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 09:16:54,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,62 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 09:16:54,214 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:16:54,214 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:16:54,215 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,215 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,215 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,216 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,215 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,216 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:54,216 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,219 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,219 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,219 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:54,219 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,219 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,219 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:54,220 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,226 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,227 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,227 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,323 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,323 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:54,323 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 09:16:54,323 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardReplica 09:16:54,323 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardReplica 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:54,325 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:54,326 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:54,327 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 09:16:54,328 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 09:16:54,328 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,328 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 09:16:54,329 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,329 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 09:16:54,329 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,329 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 09:16:54,329 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,329 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,330 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,330 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:16:54,333 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:16:54,393 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,393 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:54,393 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 09:16:54,393 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardReplica 09:16:54,393 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardReplica 09:16:54,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,396 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,396 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,396 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,397 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,397 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:54,397 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,397 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,397 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:54,397 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardReplica, member-2-shard-cars-config_testRemoveShardReplica] 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:54,399 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 09:16:54,400 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 09:16:54,400 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,400 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 09:16:54,401 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,401 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,402 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 09:16:54,402 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 09:16:54,402 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,402 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,403 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,403 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,422 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,422 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:54,422 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 09:16:54,423 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardReplica 09:16:54,423 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardReplica 09:16:54,424 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,425 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:54,425 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:54,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 09:16:54,426 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 09:16:54,427 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,427 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 09:16:54,427 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:16:54,427 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 09:16:54,427 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:54,428 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 09:16:54,428 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 09:16:54,428 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,428 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,428 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:16:54,428 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 09:16:54,428 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 09:16:54,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#986661891] 09:16:54,428 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:54,429 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:16:54,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:16:54,429 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,429 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1790959106] 09:16:54,429 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:54,429 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1282706523] 09:16:54,429 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1282706523] 09:16:54,429 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,429 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#724955445] 09:16:54,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:54,430 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] 09:16:54,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] 09:16:54,432 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-3, datastoreType Config 09:16:54,432 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:16:54,434 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:16:54,434 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 09:16:54,435 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 09:16:54,437 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'}, state: Idle 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=ac373502-ea58-40a6-8f63-359ec13cf9fb}, logIndex: 0, isSendImmediate: true 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardReplica 09:16:54,439 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 09:16:54,440 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveShardReplica'} 09:16:54,441 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]]]] 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Number of entries to be appended = 1 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, votingState=VOTING]] 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Log size is now 1 09:16:54,442 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:16:54,443 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:16:54,443 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 09:16:54,443 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 09:16:54,443 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 09:16:54,443 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 09:16:54,445 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#986661891] 09:16:54,445 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:16:54,445 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:16:54,445 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:16:54,446 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1790959106] 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardReplica updated: matchIndex: 0, nextIndex: 1 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:16:54,446 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Setting last applied to 0 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 09:16:54,446 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1282706523] 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:54,446 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1282706523] 09:16:54,446 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:16:54,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#724955445] 09:16:54,447 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:54,447 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] 09:16:54,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Commit index set to 0 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica (Follower): Setting last applied to 0 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 09:16:54,448 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=0 09:16:54,450 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] with 27000 ms timeout 09:16:54,450 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onShutDown 09:16:54,450 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardReplica 09:16:54,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 09:16:54,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 09:16:54,451 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#2118014085] 09:16:54,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:16:54,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:16:54,452 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Received ElectionTimeout 09:16:54,453 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,453 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 09:16:54,453 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardReplica 09:16:54,453 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,454 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Cluster state: [] 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,455 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:16:54,456 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:16:54,456 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:16:54,457 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 09:16:54,457 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,458 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 09:16:54,458 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:16:54,458 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,458 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:16:54,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,458 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:16:54,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:54,458 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 09:16:54,458 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 09:16:54,458 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,458 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 09:16:54,458 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:16:54,459 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 09:16:54,459 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:54,459 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:16:54,460 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,460 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,460 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardReplica updated: matchIndex: -1, nextIndex: 0 09:16:54,460 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:54,497 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#724955445] 09:16:54,497 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:16:54,497 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 09:16:54,498 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:16:54,499 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 09:16:54,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 09:16:54,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 09:16:54,500 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:16:54,500 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 09:16:54,500 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 09:16:54,501 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 09:16:54,501 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 09:16:54,502 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:16:54,502 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:16:54,502 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:16:54,502 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,503 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 09:16:54,503 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:16:54,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 09:16:54,503 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 09:16:54,503 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 09:16:54,505 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,506 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:16:54,506 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:16:54,507 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 09:16:54,508 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 09:16:54,509 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:16:54,509 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:16:54,510 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 09:16:54,510 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 09:16:54,519 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:54,519 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:54,519 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:54,519 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:16:54,522 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:16:54,554 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:16:54,567 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [2242660147488195386] 09:16:54,567 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:16:54,571 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 09:16:54,571 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:16:54,576 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:16:54,576 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:16:54,586 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:54,588 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:54,594 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:16:54,600 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1805587225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,601 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:16:54,611 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:16:54,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 09:16:54,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 09:16:54,619 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6825111f} 09:16:54,619 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 09:16:54,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:54,620 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 09:16:54,620 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:54,622 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:16:54,626 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:16:54,628 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 09:16:54,629 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardReplica, persistent : true 09:16:54,629 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,630 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-800148193] 09:16:54,630 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#712874078 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 09:16:54,630 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#712874078] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,630 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardReplica, persistent : true 09:16:54,631 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/shard-member-2:snapshot-read#-800148193] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,631 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-68190757 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 09:16:54,631 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-68419330] 09:16:54,632 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardReplica with journal batch size 1 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,634 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-982942159] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,634 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-68190757] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,634 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/shard-member-2:snapshot-read#-68419330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardReplica 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received ReplicatedLogEntry for recovery: index: 0, size: 221 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveShardReplica, address=null, votingState=VOTING]] 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Received apply journal entries for recovery, applying to state: 0 to 0 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: Moving last applied index from -1 to 0 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:54,634 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in 59.71 μs - Switching actor to Follower - last log index = 0, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 1 09:16:54,635 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:54,635 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardReplica is no longer the leader 09:16:54,635 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:16:54,635 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 09:16:54,635 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:16:54,635 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:54,636 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 09:16:54,636 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1562458356] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,637 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:16:54,637 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardReplica with journal batch size 1 09:16:54,642 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 09:16:54,642 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 09:16:54,644 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 09:16:54,645 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardReplica 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:54,649 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:54,649 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 09:16:54,649 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardReplica is no longer the leader 09:16:54,650 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:16:54,650 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:16:54,650 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 09:16:54,650 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:16:54,650 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,651 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 09:16:54,651 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3e822b14} 09:16:54,651 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:16:54,651 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,652 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardReplica, persistent : true 09:16:54,652 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardReplica with journal batch size 1 09:16:54,652 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-216282826] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,652 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/shard-member-2:snapshot-read#-425133504] 09:16:54,653 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#386533943 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 09:16:54,653 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#386533943] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardReplica 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:54,653 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:54,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:54,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardReplica is no longer the leader 09:16:54,654 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:16:54,654 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 09:16:54,654 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardReplica, persistent : true 09:16:54,655 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/shard-member-2:snapshot-read#1577590707] 09:16:54,655 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardReplica with journal batch size 1 09:16:54,655 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-884801736 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 09:16:54,655 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:54,655 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=null 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardReplica 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:16:54,655 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:16:54,655 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:16:54,656 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 09:16:54,656 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:16:54,656 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardReplica is no longer the leader 09:16:54,656 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:16:54,656 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:16:54,656 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 09:16:54,745 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 09:16:54,746 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:54,774 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 09:16:54,775 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 09:16:54,966 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:55,66 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:55,95 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:55,233 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:16:55,286 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:55,385 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:55,415 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:55,535 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5779974178252173603] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:16:55,537 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5779974178252173603] but current is [2242660147488195386]. Reason: ActorSystem terminated 09:16:55,552 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:55,592 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:55,593 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardReplica, leaderId=member-1-shard-people-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:55,594 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 09:16:55,594 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 09:16:55,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 09:16:55,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 09:16:55,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 09:16:55,594 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardReplica, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:55,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardReplica, leaderId=member-1-shard-people-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:55,595 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 09:16:55,595 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardReplica, leaderId=member-1-shard-cars-config_testRemoveShardReplica, leaderPayloadVersion=13] 09:16:55,595 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 09:16:55,596 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 09:16:55,599 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1269, lastApplied : -1, commitIndex : -1 09:16:55,600 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1170, lastApplied : -1, commitIndex : -1 09:16:55,600 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1139, lastApplied : -1, commitIndex : -1 09:16:55,600 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 1150, lastApplied : 0, commitIndex : 0 09:16:55,602 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:55,602 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:16:55,606 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:55,706 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:55,736 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:55,924 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:55,925 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:16:55,928 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 327, lastApplied : -1, commitIndex : -1 09:16:55,928 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:16:55,995 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 390, lastApplied : 0, commitIndex : 0 09:16:56,25 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:56,25 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:16:56,26 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:56,26 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 424, lastApplied : -1, commitIndex : -1 09:16:56,54 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:56,54 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:16:56,55 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:56,56 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 455, lastApplied : -1, commitIndex : -1 09:16:56,246 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:56,246 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:56,315 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:16:56,345 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:56,346 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:56,376 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,376 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,566 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,566 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,605 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:56,635 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:16:56,666 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,666 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,697 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,697 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:56,886 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,886 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:56,956 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:16:56,985 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:56,985 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:57,14 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:57,15 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:57,205 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:57,206 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:57,245 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-1505793190]] intended for system uid [5779974178252173603], self uid is [2242660147488195386] 09:16:57,276 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:16:57,306 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,306 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:57,336 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,336 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:57,479 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#960555810]] intended for system uid [5779974178252173603], self uid is [2242660147488195386] 09:16:57,526 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,527 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,595 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:16:57,625 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:57,627 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,627 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,655 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:57,656 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:57,845 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:57,846 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:57,915 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:16:57,946 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:57,946 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:57,975 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:57,975 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:58,166 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:58,167 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:58,234 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:16:58,266 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:58,266 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:58,297 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:58,297 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:58,486 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:58,487 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:58,554 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:16:58,584 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:58,584 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:58,614 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:58,614 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:16:58,644 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:58,805 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:58,805 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:16:58,874 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 09:16:58,905 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:58,905 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:58,935 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:58,935 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:59,124 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:59,125 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:59,195 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:16:59,225 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:59,225 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:16:59,255 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:59,255 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:59,444 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,444 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 09:16:59,444 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,444 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,445 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:16:59,445 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,445 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:16:59,445 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,447 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:16:59,447 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,448 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:16:59,448 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 09:16:59,448 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 09:16:59,448 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:16:59,513 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,514 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 09:16:59,514 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:59,514 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:16:59,514 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,514 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:16:59,514 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardReplica] 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,516 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 09:16:59,516 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,543 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:16:59,543 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,546 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:16:59,546 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 09:16:59,546 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-2-shard-people-config_testRemoveShardReplica, member-3-shard-people-config_testRemoveShardReplica] 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,547 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:16:59,547 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:16:59,573 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 09:16:59,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,573 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,574 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:16:59,574 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,574 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:16:59,574 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:16:59,576 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:16:59,576 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:16:59,577 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:16:59,577 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 09:16:59,577 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:16:59,577 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:16:59,664 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:16:59,765 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:59,765 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:59,836 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:16:59,864 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:59,865 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:59,895 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 09:16:59,895 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 09:17:00,85 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,86 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,155 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:17:00,186 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:00,186 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:00,215 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,215 AM [cluster-test-shard-dispatcher-20] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,333 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 09:17:00,334 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:00,334 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:00,406 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,407 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:00,475 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:17:00,505 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,505 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,535 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,535 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,578 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 09:17:00,579 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:00,579 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:00,685 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:00,725 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:00,726 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,796 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:17:00,824 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:00,825 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,855 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:00,857 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:17:01,45 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,45 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,115 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:17:01,146 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:01,147 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:17:01,175 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,175 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:01,366 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:01,367 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:01,435 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:17:01,464 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:01,464 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:17:01,494 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:01,495 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,599 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 09:17:01,601 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5779974178252173603] but current is [2242660147488195386]. Reason: Cluster member removed, previous status [Down] 09:17:01,603 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:01,603 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:01,685 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:17:01,685 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:01,704 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:01,722 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [5779974178252173603] but current is [2242660147488195386]. Reason: Cluster member removed, previous status [Down] 09:17:01,723 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:01,723 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:01,756 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:17:01,784 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,784 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,815 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:01,816 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:02,4 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,5 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:02,75 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:17:02,104 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,106 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:02,135 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:02,136 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,325 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,326 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:02,395 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:17:02,425 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:02,425 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,455 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,455 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,645 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,645 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:02,715 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:17:02,724 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:02,727 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:02,728 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:17:02,728 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:17:02,729 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1805587225] 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:17:02,729 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:17:02,730 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:17:02,730 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:02,730 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] 09:17:02,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] 09:17:02,731 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-2, datastoreType Config 09:17:02,731 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 09:17:02,732 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:02,732 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1f4eec31] 09:17:02,736 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128 for shard member-2-shard-cars-config_testRemoveShardReplica 09:17:02,736 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: onRemoveServer: RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'}, state: Idle 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: New server configuration : [ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]] 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: Returning OK for operation RemoveServer{serverId='member-2-shard-cars-config_testRemoveShardReplica'} 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 09:17:02,737 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 09:17:02,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#986661891] 09:17:02,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:02,737 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:17:02,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1460049128] 09:17:02,738 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1805587225] 09:17:02,738 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:02,738 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]]] 09:17:02,738 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Moving last applied index from 0 to 1 09:17:02,738 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] 09:17:02,738 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] 09:17:02,738 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:02,738 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: Persisting ApplyJournalEntries with index=1 09:17:02,740 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] with 27000 ms timeout 09:17:02,740 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onShutDown 09:17:02,741 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardReplica 09:17:02,741 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#127079100] 09:17:02,740 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 09:17:02,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 09:17:02,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:02,742 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:17:02,744 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,745 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,775 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:02,775 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:02,788 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1805587225] 09:17:02,788 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:02,788 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 09:17:02,789 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 09:17:02,789 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 09:17:02,789 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:17:02,789 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onShutDown 09:17:02,790 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardReplica 09:17:02,790 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:02,791 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:02,791 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 09:17:02,792 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 09:17:02,792 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 09:17:02,792 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:02,792 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onShutDown 09:17:02,792 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onShutDown 09:17:02,793 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardReplica 09:17:02,793 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardReplica 09:17:02,793 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:02,794 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:02,795 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:02,795 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:02,804 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:02,805 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:02,805 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:02,805 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:02,806 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:02,825 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:17:02,826 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 09:17:02,826 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:17:02,826 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onShutDown 09:17:02,826 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardReplica 09:17:02,827 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:02,827 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:02,827 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:17:02,827 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 09:17:02,827 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 09:17:02,827 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:02,827 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onShutDown 09:17:02,828 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onShutDown 09:17:02,828 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardReplica 09:17:02,828 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardReplica 09:17:02,828 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:02,828 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:02,829 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:02,829 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:03,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:03,813 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [2242660147488195386] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:03,843 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3435325109717566836] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:03,849 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:03,850 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:03,853 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:03.855] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:03,870 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onShutDown 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5d1e313d 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5d1e313d 09:17:03,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onShutDown 09:17:03,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: Initiating leader transfer 09:17:03,871 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 09:17:03,872 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardReplica 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28ca1efc 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@28ca1efc 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Attempting to transfer leadership 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:03,872 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 09:17:04,343 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 471.9 ms 09:17:04,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 09:17:04,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: leader transfer failed - sending PoisonPill 09:17:04,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica (Leader): Leadership transfer expired 09:17:04,344 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardReplica 09:17:04,345 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:04,345 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:04,345 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onShutDown 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Initiating leader transfer 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ff59fa0 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2ff59fa0 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:17:04,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 09:17:04,347 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onShutDown 09:17:04,347 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: Initiating leader transfer 09:17:04,347 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 09:17:04,347 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardReplica, leaderId=null, leaderPayloadVersion=13] 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1cc04be8 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1cc04be8 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Attempting to transfer leadership 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,347 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [WARN] Shard - member-1-shard-people-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveShardReplica, member-3-shard-people-oper_testRemoveShardReplica] 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 09:17:04,563 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 216.8 ms 09:17:04,564 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 09:17:04,564 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 09:17:04,564 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:04,564 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:17:04,564 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:17:04,564 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardReplica 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 346.4 ms 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer failed - sending PoisonPill 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader): Leadership transfer expired 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardReplica, member-2-shard-cars-oper_testRemoveShardReplica] 09:17:04,693 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:04,694 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:04,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:04,694 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:17:04,694 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:17:04,694 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardReplica 09:17:04,695 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:04,695 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:04,696 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:04,696 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:04,700 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:04.703] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:04,728 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:04,737 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [369609569766134360] 09:17:04,740 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:04,741 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:04,741 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:04,743 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:04,743 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:04,754 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,756 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:04,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:04,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:04,758 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:17:04,763 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,763 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:04,769 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:04,769 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,770 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,770 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:04,770 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,770 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,774 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:04,775 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:17:04,776 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,776 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7a685185} 09:17:04,776 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:04,776 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,778 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,780 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,780 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,780 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] 09:17:04,780 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,781 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveAllShardReplicas, persistent : true 09:17:04,781 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,781 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1004478895] 09:17:04,781 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#651666866 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,781 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#651666866] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,781 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,781 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,781 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-1004478895] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,781 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#293821058] 09:17:04,781 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 09:17:04,781 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveAllShardReplicas, persistent : true 09:17:04,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#293821058] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-168857875] 09:17:04,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1527296032] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-168857875] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,782 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 09:17:04,782 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:17:04,783 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-1527296032 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,784 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#597903086] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:04,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,786 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 09:17:04,786 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:04,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 09:17:04,786 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:04,786 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:04,786 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,787 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,787 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:04,787 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,787 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 09:17:04,787 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,787 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:04,789 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:17:04,789 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:17:04,791 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:04,791 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:17:04,791 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6d99c9c6} 09:17:04,792 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:04,792 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,793 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,794 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 09:17:04,794 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,794 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-660485135] 09:17:04,794 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,794 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:04,795 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1696349061 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,795 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#797941912] 09:17:04,795 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,801 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveAllShardReplicas, persistent : true 09:17:04,802 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,802 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#774392843] 09:17:04,803 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-970132895 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,803 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/shard-member-1:snapshot-read#-603571865] 09:17:04,803 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,795 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-660485135] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,803 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,803 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1696349061] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,804 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:04,804 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,804 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 09:17:04,804 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,806 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,806 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,806 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,806 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 09:17:04,806 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,804 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:04,806 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:17:04,806 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:04,806 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 09:17:04,806 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:04,806 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:04,806 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:17:04,806 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:17:04,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:04,859 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [4988239407603295249] 09:17:04,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:04,866 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:04,866 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:04,876 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:04,876 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:04,885 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,887 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,892 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:17:04,906 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:04,907 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:04,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,914 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:04,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:04,915 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 09:17:04,916 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@443877b8} 09:17:04,916 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:17:04,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:04,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:04,917 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,918 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 09:17:04,918 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveAllShardReplicas, persistent : true 09:17:04,919 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,919 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 09:17:04,919 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveAllShardReplicas, persistent : true 09:17:04,920 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 09:17:04,920 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#364557320 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:04,920 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#724213260] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,920 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1072375864] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,920 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#364557320] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,920 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1908528650] 09:17:04,921 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:17:04,921 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#294572182] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,922 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:04,924 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1605204669] 09:17:04,927 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#294572182 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 09:17:04,928 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#1908528650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,928 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-909599498] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,928 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1605204669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveAllShardReplicas is no longer the leader 09:17:04,929 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:04,929 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 09:17:04,930 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,930 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,930 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,930 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 09:17:04,930 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,930 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:04,931 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:04,932 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:17:04,936 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:04,936 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 09:17:04,936 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:17:04,936 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:04,936 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@4af8c3f0} 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:04,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,937 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:04,937 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveAllShardReplicas, persistent : true 09:17:04,938 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:04,940 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#645898390 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:04,940 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#832382347] 09:17:04,940 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#-1481788942] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,940 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#645898390] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:04,940 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 09:17:04,941 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1217096070 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,941 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/shard-member-2:snapshot-read#-1701441412] 09:17:04,941 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,941 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,941 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,941 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 09:17:04,941 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:04,941 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 09:17:04,941 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,942 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:04,942 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:17:04,942 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:04,942 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:04,942 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:04,942 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:04,942 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 09:17:04,942 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:04,942 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:04,942 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:17:04,942 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:04,942 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:17:04,952 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:04,966 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:04,967 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [4046832946543563300] 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:04,969 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:04,968 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:04,970 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:04,971 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:04,971 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:04,972 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:04,972 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:04,984 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,985 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:04,991 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:17:05,2 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,5 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:05,6 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:05,6 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,7 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,9 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:05,16 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:05,17 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:05,18 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:05,18 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,18 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,19 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@32c52a89} 09:17:05,19 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:17:05,20 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:05,21 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:05,21 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,21 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:05,22 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,22 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,22 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-3-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] 09:17:05,22 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,22 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveAllShardReplicas, persistent : true 09:17:05,22 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,23 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveAllShardReplicas with journal batch size 1 09:17:05,23 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveAllShardReplicas, persistent : true 09:17:05,23 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveAllShardReplicas with journal batch size 1 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1590315857 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#390078213] 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1074904832] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1590315857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#390078213] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1376355390 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1376355390] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,24 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-615507607] 09:17:05,25 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:17:05,25 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,25 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,25 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-3-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1074904832] 09:17:05,26 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:05,27 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-615507607] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,27 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1820695465] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,32 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,32 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,33 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,33 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,33 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:17:05,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:05,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,33 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,33 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:05,33 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,34 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,34 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:05,34 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,34 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,34 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:05,35 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7db2ea33} 09:17:05,35 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:05,36 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#117981860] 09:17:05,36 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,37 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,37 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,37 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-564035684] 09:17:05,37 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveAllShardReplicas, persistent : true 09:17:05,37 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#117981860] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,38 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-27470854 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,38 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#80012545] 09:17:05,38 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-27470854] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:05,38 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:05,39 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveAllShardReplicas, persistent : true 09:17:05,39 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-786699487 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,39 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,40 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,40 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,40 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:05,40 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 09:17:05,40 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,40 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/shard-member-3:snapshot-read#1861525829] 09:17:05,40 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,40 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:17:05,40 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:05,40 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveAllShardReplicas with journal batch size 1 09:17:05,40 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:17:05,41 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,41 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:05,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 09:17:05,41 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,41 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 09:17:05,41 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,42 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,42 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:17:05,42 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:17:05,132 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,133 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,133 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:17:05,134 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveAllShardReplicas is no longer the leader 09:17:05,153 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:17:05,153 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,202 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:17:05,202 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 09:17:05,222 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:05,222 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 09:17:05,222 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 09:17:05,222 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,222 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:17:05,223 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:05,223 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,223 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveAllShardReplicas is no longer the leader 09:17:05,223 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:17:05,223 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 09:17:05,482 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,482 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,482 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:17:05,523 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,523 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,523 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:17:05,563 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,563 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,563 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,563 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,563 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:17:05,563 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:17:05,763 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:05,763 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:05,763 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:05,763 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,764 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,812 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,812 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,812 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:17:05,812 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,813 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,814 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,815 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,815 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,815 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:05,815 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,820 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:05,820 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:05,820 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-people-oper_testRemoveAllShardReplicas, member-3-shard-people-oper_testRemoveAllShardReplicas] 09:17:05,820 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,821 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,821 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:05,821 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:17:05,821 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,822 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 09:17:05,822 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:17:05,822 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 09:17:05,822 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,822 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 09:17:05,822 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,822 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 09:17:05,823 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,823 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,823 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,823 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,882 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,882 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,882 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:17:05,882 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,883 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:05,884 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveAllShardReplicas, member-3-shard-cars-config_testRemoveAllShardReplicas] 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:05,886 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:17:05,886 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,887 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:17:05,887 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:17:05,887 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 09:17:05,889 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 09:17:05,889 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 09:17:05,889 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,889 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,890 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,892 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:05,892 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:05,893 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,932 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,932 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,932 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Received ElectionTimeout 09:17:05,932 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,932 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:17:05,932 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:17:05,932 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveAllShardReplicas 09:17:05,932 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveAllShardReplicas 09:17:05,932 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,933 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,935 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,935 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,936 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,936 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:05,937 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=null 09:17:05,937 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:05,937 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:05,937 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:05,937 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveAllShardReplicas 09:17:05,937 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:05,938 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testRemoveAllShardReplicas, member-3-shard-cars-oper_testRemoveAllShardReplicas] 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveAllShardReplicas, member-2-shard-people-config_testRemoveAllShardReplicas] 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:05,938 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:17:05,938 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:17:05,938 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,938 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 09:17:05,938 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 09:17:05,938 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:05,938 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:17:05,938 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 09:17:05,938 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] 09:17:05,939 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:05,940 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] 09:17:05,940 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] 09:17:05,941 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 09:17:05,941 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 09:17:05,941 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 09:17:05,941 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 09:17:05,941 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,941 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,941 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveAllShardReplicas, leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,941 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:05,941 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] 09:17:05,941 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:17:05,941 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:17:05,941 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveAllShardReplicas, leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:05,941 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:05,941 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:05,941 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#724213260] 09:17:05,942 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#724213260] 09:17:05,942 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] 09:17:05,942 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:05,942 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] 09:17:05,942 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,942 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,942 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,942 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] 09:17:05,942 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,943 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,943 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:05,943 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:05,953 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@55bd45cc] 09:17:05,954 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:17:05,956 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Shard created : member-1-shard-pets-config_testRemoveAllShardReplicas, persistent : true 09:17:05,957 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1763747010 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:05,957 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/shard-member-1:snapshot-read#1866609641] 09:17:05,957 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Starting recovery for member-1-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 09:17:05,958 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,958 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,958 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 09:17:05,958 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 09:17:05,958 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,958 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,959 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,959 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:05,959 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:17:05,968 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@892cf38] 09:17:05,969 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:17:05,970 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-2-shard-pets-config_testRemoveAllShardReplicas, persistent : true 09:17:05,971 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/shard-member-2:snapshot-read#604495631] 09:17:05,971 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#1921700532 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:05,971 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-2-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 09:17:05,972 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,972 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,972 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 09:17:05,972 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 09:17:05,972 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,972 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-2-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,972 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,972 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:05,972 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:17:05,981 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5204c1a0] 09:17:05,982 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=null, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:17:05,983 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] 09:17:05,983 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:17:05,984 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#354735312] 09:17:05,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#354735312] 09:17:05,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] 09:17:05,984 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-3-shard-pets-config_testRemoveAllShardReplicas, persistent : true 09:17:05,984 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:17:05,984 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1628913151] 09:17:05,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1628913151] 09:17:05,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] 09:17:05,984 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 09:17:05,984 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-1254873552 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:05,985 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/shard-member-3:snapshot-read#-1911491339] 09:17:05,985 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-3-shard-pets-config_testRemoveAllShardReplicas with journal batch size 1 09:17:05,988 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:05,988 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:05,988 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:05,988 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-3-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 09:17:05,988 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:17:05,985 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 09:17:05,998 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 09:17:05,998 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 09:17:05,998 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 09:17:05,998 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-107464134] 09:17:05,998 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:17:05,998 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-107464134] 09:17:05,998 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:17:06,0 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 09:17:06,0 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 09:17:06,0 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 09:17:06,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 09:17:06,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:17:06,1 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 09:17:06,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 09:17:06,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:06,2 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 09:17:06,2 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:06,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'}, state: Idle 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]] 09:17:06,3 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'}, state: Idle 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'}, state: Idle 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]] 09:17:06,3 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]] 09:17:06,3 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=d093a91d-4055-4ea6-8370-3476b608b95f}, logIndex: 0, isSendImmediate: true 09:17:06,3 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,3 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,3 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8c439b36-df22-4fc0-900d-1bf5e27669ef}, logIndex: 0, isSendImmediate: true 09:17:06,4 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=12eac9c5-eada-4069-84a0-eb4f40ddc3ed}, logIndex: 0, isSendImmediate: true 09:17:06,4 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-oper_testRemoveAllShardReplicas'} 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:06,4 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,4 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-oper_testRemoveAllShardReplicas'} 09:17:06,4 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-cars-config_testRemoveAllShardReplicas'} 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'}, state: Idle 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=89835e01-b01d-44e3-b7a7-cf3167c3a0d3}, logIndex: 0, isSendImmediate: true 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testRemoveAllShardReplicas 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-people-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-people-config_testRemoveAllShardReplicas'} 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 09:17:06,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 09:17:06,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:17:06,6 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 09:17:06,6 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,6 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 09:17:06,6 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-pets-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-pets-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:06,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:17:06,7 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-pets-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,7 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:06,7 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:17:06,7 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,7 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,7 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, votingState=VOTING]] 09:17:06,7 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 09:17:06,7 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-oper_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, votingState=VOTING]] 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Log size is now 1 09:17:06,7 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, votingState=VOTING]] 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Log size is now 1 09:17:06,8 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 09:17:06,8 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:06,8 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:06,8 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,8 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,8 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:06,8 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-people-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, votingState=VOTING]] 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Setting last applied to 0 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Log size is now 1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,9 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:06,9 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,10 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 09:17:06,10 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#117981860] with 27000 ms timeout 09:17:06,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 09:17:06,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:06,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] with 27000 ms timeout 09:17:06,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 09:17:06,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, people] 09:17:06,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Commit index set to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:06,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Setting last applied to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,11 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:06,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:17:06,11 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,11 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onShutDown 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:06,12 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,12 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveAllShardReplicas 09:17:06,11 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-564035684] with 27000 ms timeout 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onShutDown 09:17:06,12 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1616099618] 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:06,12 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 09:17:06,11 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,13 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveAllShardReplicas 09:17:06,13 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#117981860] 09:17:06,11 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,13 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,13 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-564035684] 09:17:06,13 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 09:17:06,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1074904832] with 27000 ms timeout 09:17:06,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 09:17:06,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 09:17:06,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onShutDown 09:17:06,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:06,14 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveAllShardReplicas 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 09:17:06,14 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1074904832] 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,14 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:17:06,282 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Up), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Up)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:06,282 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower): Received ElectionTimeout - switching to Candidate 09:17:06,282 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Election: Candidate has following voting peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 09:17:06,282 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:06,282 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:17:06,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Sending RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,283 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:06,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:17:06,283 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard member-1-shard-pets-config_testRemoveAllShardReplicas is no longer the leader 09:17:06,284 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:17:06,284 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Cluster state: [] 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Candidate in requestVote:RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=null 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:06,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:06,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): In requestVote: RequestVote [term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 1, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:06,286 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Set currentTerm=1, votedFor=member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:06,286 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 09:17:06,286 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): requestVote returning: RequestVoteReply [term=1, voteGranted=true] 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): RaftRPC message received RequestVoteReply [term=1, voteGranted=true], my term is 1 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): handleRequestVoteReply: RequestVoteReply [term=1, voteGranted=true], current voteCount: 1 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Election: Leader has following peers: [member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas] 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:17:06,287 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:17:06,288 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:17:06,288 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:06,288 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 09:17:06,288 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:06,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 09:17:06,289 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 09:17:06,289 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:06,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderPayloadVersion=13] 09:17:06,289 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 09:17:06,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:06,289 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 09:17:06,289 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:17:06,290 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,291 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: -1, nextIndex: 0 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-3-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onRemoveServer: RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'}, state: Idle 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: New server configuration : [ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]] 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=154cb63a-b3cb-4348-bc22-da29254a4a7e}, logIndex: 0, isSendImmediate: true 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): sendAppendEntries: 0 is present for follower member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,292 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Sending AppendEntries to follower member-2-shard-pets-config_testRemoveAllShardReplicas: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,293 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: Returning OK for operation RemoveServer{serverId='member-3-shard-pets-config_testRemoveAllShardReplicas'} 09:17:06,293 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,294 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 09:17:06,294 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-pets-config_testRemoveAllShardReplicas, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]]]] 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Number of entries to be appended = 1 09:17:06,294 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard pets, type Config 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,294 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-pets-config_testRemoveAllShardReplicas, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, votingState=VOTING]] 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Log size is now 1 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-pets-config_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:06,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]]] 09:17:06,295 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 09:17:06,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-pets-config_testRemoveAllShardReplicas updated: matchIndex: 0, nextIndex: 1 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Setting last applied to 0 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:06,295 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:06,295 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:06,295 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] 09:17:06,296 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-466187733] 09:17:06,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] 09:17:06,296 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:06,296 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#293821058] 09:17:06,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#293821058] 09:17:06,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1425668196] 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Commit index set to 0 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Setting last applied to 0 09:17:06,297 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Moving last applied index from -1 to 0 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]] 09:17:06,297 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#354735312] 09:17:06,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#354735312] 09:17:06,297 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Persisting ApplyJournalEntries with index=0 09:17:06,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-107464134] with 27000 ms timeout 09:17:06,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 09:17:06,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 09:17:06,297 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onShutDown 09:17:06,297 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:06,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] 09:17:06,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:06,298 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-3-shard-pets-config_testRemoveAllShardReplicas 09:17:06,298 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:17:06,298 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#724213260] 09:17:06,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#724213260] 09:17:06,298 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-107464134] 09:17:06,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] 09:17:06,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:17:06,298 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1072375864] 09:17:06,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-1072375864] 09:17:06,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1127448857] 09:17:06,298 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:17:06,298 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1628913151] 09:17:06,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1628913151] 09:17:06,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 09:17:06,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 09:17:06,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#967017885] 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 09:17:06,299 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 09:17:06,299 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:06,299 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:06,299 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:06,300 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 09:17:06,300 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:06,300 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:06,301 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:06,301 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:06,306 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:06,306 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:06,306 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:06,306 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:06,308 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:06,317 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:06,318 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,318 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:06,318 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 09:17:06,318 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 09:17:06,318 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onShutDown 09:17:06,318 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onShutDown 09:17:06,318 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onShutDown 09:17:06,319 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveAllShardReplicas 09:17:06,319 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveAllShardReplicas 09:17:06,319 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-pets-config_testRemoveAllShardReplicas 09:17:06,319 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:06,319 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:06,320 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onShutDown 09:17:06,320 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 09:17:06,320 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveAllShardReplicas 09:17:06,321 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:06,321 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:06,323 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:06,323 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:06,782 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:07,323 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [4046832946543563300] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:07,340 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:07,341 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:07,343 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [4988239407603295249] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:07,343 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:07.345] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:07,357 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onShutDown 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onShutDown 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Initiating leader transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Initiating leader transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c12443e 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1c12443e 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@193aba48 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@193aba48 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-pets-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onShutDown 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Initiating leader transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dcb9c8a 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5dcb9c8a 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-pets-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 09:17:07,358 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:07,358 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 09:17:07,802 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 09:17:07,802 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 444.4 ms 09:17:07,803 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 09:17:07,803 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 09:17:07,803 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 09:17:07,803 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveAllShardReplicas 09:17:07,852 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 09:17:07,852 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 493.7 ms 09:17:07,852 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 09:17:07,852 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 09:17:07,852 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 09:17:07,853 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveAllShardReplicas 09:17:07,882 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 09:17:07,882 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 524.4 ms 09:17:07,882 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 09:17:07,882 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 09:17:07,882 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas (Leader): Leadership transfer expired 09:17:07,883 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - Stopping Shard member-1-shard-pets-config_testRemoveAllShardReplicas 09:17:07,883 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:07,883 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:07,884 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:17:07,884 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:07,884 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:07,884 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:07,884 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onShutDown 09:17:07,884 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Initiating leader transfer 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onShutDown 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Initiating leader transfer 09:17:07,885 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c38270b 09:17:07,885 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c38270b 09:17:07,885 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:17:07,885 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 09:17:07,885 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 09:17:07,885 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a865409 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a865409 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Attempting to transfer leadership 09:17:07,885 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveAllShardReplicas, leaderId=null, leaderPayloadVersion=13] 09:17:07,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveAllShardReplicas: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:08,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 09:17:08,372 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 487.2 ms 09:17:08,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 09:17:08,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 09:17:08,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 09:17:08,373 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveAllShardReplicas 09:17:08,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 09:17:08,493 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 608.0 ms 09:17:08,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 09:17:08,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer failed - sending PoisonPill 09:17:08,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader): Leadership transfer expired 09:17:08,494 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 09:17:08,494 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:08,494 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:08,496 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:08,496 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [11/03/2024 09:17:08.501] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:08,529 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:17:08,540 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6680157580074972013] 09:17:08,540 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:08,544 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:08,544 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:08,546 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:08,546 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:08,556 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,560 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:08,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:08,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:08,562 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 09:17:08,568 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:08,568 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#502367491] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,571 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:08,571 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,571 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,571 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,590 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 09:17:08,591 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@50adbbe9} 09:17:08,591 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 09:17:08,591 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,591 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,591 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,593 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,593 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,593 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#308940913] 09:17:08,593 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplicaFailures, persistent : true 09:17:08,594 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#308940913] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,594 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-581780706 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,594 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-581780706] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,594 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,595 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplicaFailures with journal batch size 1 09:17:08,595 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1423519511] 09:17:08,595 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/shard-member-1:snapshot-read#1423519511] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,598 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 09:17:08,600 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:08,602 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:08,602 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:08,602 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#1262329845] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,602 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:08,602 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,603 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,603 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,603 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:08,603 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 09:17:08,605 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplicaFailures is no longer the leader 09:17:08,605 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 09:17:08,605 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 09:17:08,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 09:17:08,606 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 09:17:08,606 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderId=member-1-shard-cars-config_testAddShardReplicaFailures, leaderPayloadVersion=13] 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 09:17:08,606 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1f5755b7} 09:17:08,606 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 09:17:08,608 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,608 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,609 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,609 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,609 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#143610816] 09:17:08,609 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplicaFailures, persistent : true 09:17:08,610 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplicaFailures with journal batch size 1 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#143610816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-2110609377] 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-534238041 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-534238041] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,610 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard people 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/shard-member-1:snapshot-read#-2110609377] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,610 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 09:17:08,610 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 09:17:08,610 AM [cluster-test-pekko.actor.default-dispatcher-6] [ERROR] ClusterAdminRpcService - Failed to add replica for shard people java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1194) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:247) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:17:08,610 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:08,611 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 09:17:08,611 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 09:17:08,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:08,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower): Received TimeoutNow - switching to Candidate 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Election: Candidate has following voting peers: [] 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:08,611 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplicaFailures is no longer the leader 09:17:08,611 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 09:17:08,611 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:17:08,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onShutDown 09:17:08,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@114f51f6 09:17:08,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@114f51f6 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Received ElectionTimeout 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Leader): Election: Leader has following peers: [] 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 09:17:08,612 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 09:17:08,612 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 09:17:08,612 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplicaFailures 09:17:08,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderId=member-1-shard-cars-oper_testAddShardReplicaFailures, leaderPayloadVersion=13] 09:17:08,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 09:17:08,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 09:17:08,612 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:08,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:08,612 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:17:08,613 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f38f07eba30] to Actor[pekko://cluster-test/user/$a#-558793024] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-558793024] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onShutDown 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5a27b6e6 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5a27b6e6 09:17:08,613 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#502367491] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#502367491] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,613 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplicaFailures 09:17:08,613 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:08,614 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:08,615 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:08,615 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:08,616 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:08,631 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:08,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-7130990352757433365] 09:17:08,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:08,641 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:08,641 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:08,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:08,642 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:08,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:08,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:08,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:08,657 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,659 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,661 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:17:08,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:08,669 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1903334393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,670 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:08,670 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 09:17:08,670 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,670 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:08,673 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:08,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@32b1e00d} 09:17:08,673 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 09:17:08,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 09:17:08,676 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:08,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:08,676 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553] 09:17:08,677 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,677 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-cars-config_testAddShardReplica, persistent : true 09:17:08,677 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#294823015 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 09:17:08,677 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#294823015] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,678 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#-1730842298] 09:17:08,678 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/shard-member-1:snapshot-read#-1730842298] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,678 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testAddShardReplica with journal batch size 1 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:08,679 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:08,679 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 09:17:08,679 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testAddShardReplica 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testAddShardReplica is no longer the leader 09:17:08,679 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 09:17:08,680 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate): Received ElectionTimeout 09:17:08,680 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Election: Leader has following peers: [] 09:17:08,680 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:08,680 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:08,680 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 09:17:08,680 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 09:17:08,680 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:17:08,680 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 09:17:08,680 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 09:17:08,680 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:17:08,724 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1903334393] 09:17:08,724 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:08,724 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553] 09:17:08,724 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,726 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,728 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:17:08,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1950670066] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,729 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:08,730 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:08,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 09:17:08,730 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,730 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:08,734 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:08,734 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3c76182b} 09:17:08,734 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:17:08,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 09:17:08,735 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:08,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:08,735 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] 09:17:08,736 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-cars-oper_testAddShardReplica, persistent : true 09:17:08,736 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,737 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testAddShardReplica with journal batch size 1 09:17:08,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-1413275688] 09:17:08,737 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/shard-member-1:snapshot-read#-1413275688] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,737 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1922798442 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 09:17:08,737 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1922798442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,737 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:08,737 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:08,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:08,737 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower): Received TimeoutNow - switching to Candidate 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Election: Candidate has following voting peers: [] 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testAddShardReplica 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:08,738 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:17:08,738 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate): Received ElectionTimeout 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Election: Leader has following peers: [] 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 09:17:08,738 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 09:17:08,738 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 09:17:08,738 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:08,785 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1950670066] 09:17:08,785 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:08,785 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] 09:17:08,790 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:08,798 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-6658571468512167435] 09:17:08,799 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:08,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:08,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:08,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:08,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:08,814 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,816 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,818 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:17:08,825 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1513833301] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,827 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:08,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:08,833 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:08,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@763113be} 09:17:08,834 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 09:17:08,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,835 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,837 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:08,839 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:17:08,841 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:08,841 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:08,843 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:08,844 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:08,844 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:17:08,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3333831c} 09:17:08,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:08,851 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:08,856 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,856 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:08,856 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:08,856 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:17:08,856 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:09,656 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:09,658 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,658 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,658 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:09,658 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:17:09,658 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:09,658 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:09,658 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,660 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:09,748 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:17:09,748 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:17:09,748 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:09,748 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 09:17:09,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:17:09,750 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553, primaryVersion=13] 09:17:09,753 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553 for shard member-2-shard-cars-config_testAddShardReplica 09:17:09,754 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-cars-config_testAddShardReplica, persistent : true 09:17:09,754 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,755 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-577711310] 09:17:09,755 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-766987279 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 09:17:09,755 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-766987279] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,755 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 09:17:09,755 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/shard-member-2:snapshot-read#-577711310] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,755 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:09,755 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testAddShardReplica with journal batch size 1 09:17:09,755 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:17:09,755 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 09:17:09,755 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 1 09:17:09,756 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:17:09,756 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-config_testAddShardReplica 09:17:09,756 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 09:17:09,756 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:09,756 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:09,756 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@713a41e7, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=null] 09:17:09,756 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:09,757 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:17:09,757 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 09:17:09,757 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 09:17:09,757 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:17:09,757 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:17:09,757 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-config_testAddShardReplica: 229 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-2-shard-cars-config_testAddShardReplica), Chunk: 1/1 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 09:17:09,757 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 09:17:09,758 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:17:09,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:09,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:17:09,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:17:09,758 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 09:17:09,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@40b92bc, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to -1 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 09:17:09,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2010922923] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 336.7 μs 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: SnapshotComplete received 09:17:09,760 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2010922923] 09:17:09,760 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:09,760 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 67.90 μs 09:17:09,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a snapshot response 09:17:09,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: handled a journal response 09:17:09,761 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2010922923] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-config_testAddShardReplica] 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:09,761 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=f1ca8be8-d9a3-4128-b622-637ef446af39}, logIndex: 0, isSendImmediate: true 09:17:09,762 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:09,762 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testAddShardReplica 09:17:09,762 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:09,762 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 09:17:09,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:09,763 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553 09:17:09,763 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:17:09,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 09:17:09,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:17:09,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 09:17:09,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:17:09,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testAddShardReplica is no longer the leader 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 09:17:09,764 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:09,764 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1513833301] 09:17:09,763 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Successfully deleted prior snapshots 09:17:09,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:09,765 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:09,765 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 0 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:09,765 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 09:17:09,765 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] 09:17:09,766 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:09,766 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 09:17:09,766 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:17:09,766 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 0 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 0 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=0 09:17:09,766 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:09,766 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 09:17:09,766 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 09:17:09,767 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:17:09,767 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173, primaryVersion=13] 09:17:09,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173 for shard member-2-shard-cars-oper_testAddShardReplica 09:17:09,770 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,770 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-cars-oper_testAddShardReplica, persistent : true 09:17:09,771 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#-95163554] 09:17:09,771 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1426042532 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 09:17:09,771 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#-1426042532] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,771 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/shard-member-2:snapshot-read#-95163554] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,771 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testAddShardReplica with journal batch size 1 09:17:09,771 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex -1 and lastAppliedTerm -1 instead. 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 1 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-2-shard-cars-oper_testAddShardReplica 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3ec0bb8b, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=null] 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:17:09,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-2-shard-cars-oper_testAddShardReplica: 229 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica), Chunk: 1/1 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 09:17:09,773 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:09,773 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:09,773 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:09,773 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null]" message is greater than follower's term 0 - updating term 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:09,773 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 09:17:09,774 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3b5f7176, electionTerm=1, electionVotedFor=null, ServerConfigPayload=null] 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 09:17:09,774 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to -1 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 09:17:09,774 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:17:09,774 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 09:17:09,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2120928346] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:17:09,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2120928346] 09:17:09,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 267.3 μs 09:17:09,775 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 09:17:09,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:17:09,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 09:17:09,775 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: SnapshotComplete received 09:17:09,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 62.37 μs 09:17:09,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2120928346] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,775 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 09:17:09,775 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:09,775 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a snapshot response 09:17:09,775 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: handled a journal response 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-2-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-2-shard-cars-oper_testAddShardReplica] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=9f764d28-50e4-450e-9a4e-59921e9ae8b5}, logIndex: 0, isSendImmediate: true 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testAddShardReplica 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-2-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:09,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 09:17:09,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=0, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 09:17:09,777 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:17:09,777 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:09,778 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=0, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:09,778 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:17:09,778 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] 09:17:09,778 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 09:17:09,778 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:17:09,778 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:09,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 0, nextIndex: 1 09:17:09,778 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:09,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 0 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:09,778 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Successfully deleted prior snapshots 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:09,778 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:09,778 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:09,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=0 09:17:09,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 09:17:09,789 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:17:09,800 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-6750405162752676084] 09:17:09,801 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:09,805 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:09,805 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:09,805 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:09,805 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:09,811 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:09,812 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:09,814 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:17:09,822 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,822 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:17:09,822 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:17:09,822 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:09,822 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,822 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:09,822 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:09,822 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,824 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:09,828 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1510509372] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,828 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:09,831 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:09,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@4ecd6bc} 09:17:09,833 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 09:17:09,833 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:09,834 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:09,835 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:09,837 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:17:09,838 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#2072502548] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:09,838 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:09,840 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:09,842 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:09,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@a2b2851} 09:17:09,844 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:17:09,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:09,855 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:09,857 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:09,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:09,859 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:09,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,859 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:09,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,859 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:09,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:09,859 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:10,87 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 09:17:10,387 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 09:17:11,697 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:11,698 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:11,698 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:11,698 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:11,698 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:11,698 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:17:11,698 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:11,698 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:11,698 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:12,716 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:12,718 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:12,718 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:12,718 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:17:12,718 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:12,718 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:12,718 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:12,718 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:12,718 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:17:12,750 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:17:12,750 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:17:12,751 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,751 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 09:17:12,753 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,753 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553, primaryVersion=13] 09:17:12,756 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553 for shard member-3-shard-cars-config_testAddShardReplica 09:17:12,757 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 09:17:12,757 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#370096183] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,758 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-2024121932 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 09:17:12,758 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#-2024121932] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,758 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1996932653] 09:17:12,758 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 09:17:12,758 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#-1996932653] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: lastSequenceNumber prior to capture: 3 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-config_testAddShardReplica 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: CaptureSnapshotReply received by actor 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@38498b7d, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:17:12,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendInstallSnapshot 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-config_testAddShardReplica: 229 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica), Chunk: 1/1 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 3 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: SnapshotComplete received 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a journal response 09:17:12,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: handled a snapshot response 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 09:17:12,761 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:12,761 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 09:17:12,761 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:17:12,761 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-config_testAddShardReplica 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@292d5482, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:17:12,762 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=-1] 09:17:12,762 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:12,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Snapshot success - sequence number: 1 09:17:12,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 09:17:12,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:12,762 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1411625897] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 377.0 μs 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotComplete received 09:17:12,763 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1411625897] 09:17:12,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a journal response 09:17:12,764 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1411625897] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: handled a snapshot response 09:17:12,764 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:12,764 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 70.41 μs 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-config_testAddShardReplica, chunkIndex=1, success=true] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-config_testAddShardReplica] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=bc36ded4-3c47-465b-97d4-f017e13dc4cf}, logIndex: 1, isSendImmediate: true 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testAddShardReplica 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:12,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:12,767 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testAddShardReplica 09:17:12,767 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:12,767 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-config_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:12,767 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, votingState=VOTING]] 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 1 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 09:17:12,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 09:17:12,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]]] 09:17:12,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:17:12,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-config_testAddShardReplica 09:17:12,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:17:12,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 09:17:12,769 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:17:12,769 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1510509372] 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 1 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:17:12,769 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:12,770 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#370096183] 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 09:17:12,770 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#370096183] 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 09:17:12,770 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#2072502548] 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:17:12,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:12,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:12,770 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard cars 09:17:12,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:17:12,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,770 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Successfully deleted prior snapshots 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 09:17:12,771 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,772 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173, primaryVersion=13] 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 1 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 1 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]] 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,772 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:17:12,774 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-oper_testAddShardReplica, persistent : true 09:17:12,775 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-475304859] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173 for shard member-3-shard-cars-oper_testAddShardReplica 09:17:12,775 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-891733850 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 09:17:12,775 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-891733850] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,775 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-284842804] 09:17:12,775 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/shard-member-3:snapshot-read#-284842804] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,776 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testAddShardReplica with journal batch size 1 09:17:12,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onAddServer: AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true], state: Idle 09:17:12,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:12,776 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 09:17:12,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: lastSequenceNumber prior to capture: 3 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Initiating capture snapshot for new server member-3-shard-cars-oper_testAddShardReplica 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: CaptureSnapshotReply received by actor 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6548a705, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testAddShardReplica, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snaphsotIndex: -1 and term: -1 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendInstallSnapshot 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:12,777 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): next snapshot chunk size for follower member-3-shard-cars-oper_testAddShardReplica: 229 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-oper_testAddShardReplica, member-3-shard-cars-oper_testAddShardReplica), Chunk: 1/1 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:12,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 3 09:17:12,777 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: SnapshotComplete received 09:17:12,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a snapshot response 09:17:12,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: handled a journal response 09:17:12,778 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:17:12,778 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 09:17:12,778 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]" message is greater than follower's term 0 - updating term 09:17:12,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:12,778 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot: InstallSnapshot [term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,779 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:17:12,779 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot installed from leader: member-1-shard-cars-oper_testAddShardReplica 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=-1] 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: Snapshot [lastIndex=0, lastTerm=1, lastAppliedIndex=0, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7a978856, electionTerm=1, electionVotedFor=null, ServerConfigPayload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - lastSequenceNumber prior to persisting applied snapshot: 1 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: SaveSnapshotSuccess received for snapshot, sequenceNr: 1 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Snapshot success - sequence number: 1 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from -1 to 0 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 09:17:12,779 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1241271564] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 339.4 μs 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: SnapshotComplete received 09:17:12,780 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1241271564] 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a journal response 09:17:12,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: handled a snapshot response 09:17:12,780 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:17:12,780 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 75.18 μs 09:17:12,781 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1241271564] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=1, followerId=member-3-shard-cars-oper_testAddShardReplica, chunkIndex=1, success=true] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - Sent message UnInitializedFollowerSnapshotReply to self 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: onUnInitializedFollowerSnapshotReply: UnInitializedFollowerSnapshotReply [followerId=member-3-shard-cars-oper_testAddShardReplica] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8d956136-45ba-44e3-95dd-d2ff4df4f2c6}, logIndex: 1, isSendImmediate: true 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testAddShardReplica 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testAddShardReplica 09:17:12,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=4, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:12,782 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: Returning OK for operation AddServer [newServerId=member-3-shard-cars-oper_testAddShardReplica, newServerAddress=null, votingMember=true] 09:17:12,782 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 09:17:12,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=0, prevLogTerm=1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]]]] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testAddShardReplica, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, votingState=VOTING]] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 1 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=1, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173 09:17:12,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 09:17:12,783 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=1, term=1, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]]] 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 1 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 1, nextIndex: 2 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testAddShardReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:12,784 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 09:17:12,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 1 09:17:12,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 09:17:12,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 09:17:12,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 0 to 1 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]] 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testAddShardReplica, leaderId=member-1-shard-cars-oper_testAddShardReplica, leaderPayloadVersion=13] 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 0 for term 1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=1 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:17:12,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Re-initializing to Follower with previous leaderId member-1-shard-cars-oper_testAddShardReplica 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:12,786 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Successfully deleted prior snapshots 09:17:12,786 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully added replica for shard cars 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#2072502548] 09:17:12,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:12,787 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-475304859] 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-475304859] 09:17:12,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:17:12,787 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testAddShardReplica is no longer the leader 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1513833301] 09:17:12,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:12,787 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] 09:17:12,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:12,787 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:12,787 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:12,835 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1513833301] 09:17:12,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:12,835 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:12,847 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 09:17:12,858 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=63401, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:12,867 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:12,871 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:17:12,874 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:17:12,874 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:12,894 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 21.79 ms 09:17:12,921 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,938 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,938 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 09:17:12,942 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 09:17:12,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 09:17:12,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 09:17:12,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:17:12,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 09:17:12,947 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,950 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 09:17:12,950 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:17:12,953 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 09:17:12,953 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:17:12,953 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 09:17:12,953 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:17:12,953 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 09:17:12,954 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,958 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 09:17:12,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 09:17:12,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:17:12,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testAddShardReplica 09:17:12,960 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 09:17:12,960 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:17:12,960 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testAddShardReplica 09:17:12,960 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 09:17:12,960 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 09:17:12,964 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 09:17:12,965 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,965 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}]]] 09:17:12,965 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:12,965 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log Simple{size=235} 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:12,966 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=235}] 09:17:12,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 09:17:12,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:17:12,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 2 09:17:12,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 09:17:12,966 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:17:12,966 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 09:17:12,970 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,971 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@44c74d0c}, purged=MutableUnsignedLongSet{size=0}} 09:17:12,971 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:12,973 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,973 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:17:12,973 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:17:12,974 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-366556560] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 09:17:12,975 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-366556560] 09:17:12,975 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:12,975 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-366556560] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:12,975 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 108.3 μs 09:17:12,983 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1510509372] 09:17:12,983 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:12,983 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:12,983 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:12,989 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:12,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:17:12,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 09:17:12,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:17:12,990 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:12,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 09:17:12,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:17:12,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 09:17:12,990 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 2 09:17:12,990 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 09:17:12,991 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 09:17:12,991 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:17:12,991 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 2 09:17:12,991 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 09:17:12,991 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 2 data Simple{size=235} 09:17:12,991 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,991 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,993 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@14a35bdd}, purged=MutableUnsignedLongSet{size=0}} 09:17:12,992 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7c961791}, purged=MutableUnsignedLongSet{size=0}} 09:17:12,993 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:12,993 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:12,993 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,993 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,993 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:12,993 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:12,993 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 100.2 μs 09:17:12,993 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 84.36 μs 09:17:12,993 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:12,993 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:12,993 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:12,994 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:12,994 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:12,996 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:12,996 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:12,997 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Replicate message: identifier: member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-config_testAddShardReplica 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-config_testAddShardReplica 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 09:17:12,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=13410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 09:17:13,0 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,0 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Setting last applied to 3 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 09:17:13,1 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f38f099f4b8@3abd8a09 09:17:13,2 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}]]] 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Log size is now 2 09:17:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-config_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,3 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:17:13,3 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:17:13,3 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56}] 09:17:13,3 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,3 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 649.8 μs 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:13,5 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,9 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Commit index set to 3 09:17:13,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:17:13,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica (Follower): Setting last applied to 3 09:17:13,10 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1, size=56} 09:17:13,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,11 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,12 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,19 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#2072502548] 09:17:13,19 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:13,19 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:17:13,19 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:17:13,21 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 09:17:13,21 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=13800, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:13,22 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:17:13,22 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:17:13,24 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.923 ms 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 09:17:13,26 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 2, isSendImmediate: true 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testAddShardReplica 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testAddShardReplica 09:17:13,27 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 09:17:13,28 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 09:17:13,28 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 09:17:13,28 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=1, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}]]] 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,28 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,28 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:13,28 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log Simple{size=233} 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 09:17:13,28 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,29 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 09:17:13,29 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=2, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,29 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,29 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=2, term=1, payload=Simple{size=233}] 09:17:13,29 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 2 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 09:17:13,29 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@18c0688e}, purged=MutableUnsignedLongSet{size=0}} 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:17:13,30 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#2090176981] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:17:13,30 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#2090176981] 09:17:13,31 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:17:13,31 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 117.0 μs 09:17:13,31 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] 09:17:13,31 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#2090176981] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,31 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:13,31 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:17:13,31 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:17:13,32 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:13,33 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:17:13,33 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 2, nextIndex: 3 09:17:13,33 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:17:13,33 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:13,33 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:17:13,33 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 09:17:13,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:17:13,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 1 to 2 09:17:13,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 2 data Simple{size=233} 09:17:13,34 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,34 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persist data SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,34 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,34 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Replicate message: identifier: member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, logIndex: 3, isSendImmediate: false 09:17:13,34 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 09:17:13,34 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=9870, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:13,34 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@559e51e6}, purged=MutableUnsignedLongSet{size=0}} 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,34 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:13,34 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 95.50 μs 09:17:13,34 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@432a22f2}, purged=MutableUnsignedLongSet{size=0}} 09:17:13,34 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 09:17:13,34 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,35 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:17:13,35 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 84.51 μs 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 1 for term 1 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=2 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-oper_testAddShardReplica 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-oper_testAddShardReplica 09:17:13,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testAddShardReplica: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 09:17:13,36 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:17:13,36 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:17:13,36 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 444.1 μs 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-3-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testAddShardReplica, prevLogIndex=2, prevLogTerm=1, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}]]] 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Number of entries to be appended = 1 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Append entry to log PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Log size is now 2 09:17:13,37 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=1, success=true, followerId=member-2-shard-cars-oper_testAddShardReplica, logLastIndex=3, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:13,38 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: persist complete SimpleReplicatedLogEntry [index=3, term=1, payload=PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54}] 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Setting last applied to 3 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f38f099f4b8@c1de00d 09:17:13,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,39 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,39 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,39 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,39 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1903334393] 09:17:13,39 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:13,39 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553] 09:17:13,39 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553] 09:17:13,40 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1950670066] 09:17:13,40 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:13,40 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] 09:17:13,40 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] 09:17:13,40 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:13,40 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testAddShardReplica updated: matchIndex: 3, nextIndex: 4 09:17:13,40 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,41 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Commit index set to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica (Follower): Setting last applied to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1, size=54} 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:13,41 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: Persisting ApplyJournalEntries with index=3 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1513833301] 09:17:13,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:13,42 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-688990832] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1543055114] 09:17:13,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:13,42 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#-860433281] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1510509372] 09:17:13,42 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:13,42 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#370096183] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#370096183] 09:17:13,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#2072502548] 09:17:13,42 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:13,43 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-475304859] 09:17:13,43 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-475304859] 09:17:13,43 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:13,43 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 09:17:13,43 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 09:17:13,43 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:17:13,43 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 09:17:13,44 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 09:17:13,44 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:13,44 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:13,45 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:13,45 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 09:17:13,45 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 09:17:13,45 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onShutDown 09:17:13,45 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testAddShardReplica 09:17:13,46 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:13,46 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:13,46 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:13,46 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:13,48 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 09:17:13,48 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 09:17:13,48 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 09:17:13,48 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 09:17:13,53 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:13,53 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:13,53 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:13,53 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:13,54 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:13,73 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:13,83 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-659586281271436772] 09:17:13,83 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:13,86 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:13,86 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:13,87 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:13,87 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:13,96 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:13,100 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:13,106 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:17:13,112 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1036404327] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,113 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:13,114 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onSnapshotOffer: ShardManagerSnapshot [ShardList = [cars] ] 09:17:13,114 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 09:17:13,115 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:13,115 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 09:17:13,122 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1935118c} 09:17:13,122 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 09:17:13,123 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1036404327] 09:17:13,123 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:13,123 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 09:17:13,123 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:13,123 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 09:17:13,123 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testAddShardReplica, persistent : true 09:17:13,124 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testAddShardReplica with journal batch size 1 09:17:13,124 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-877416096] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#256794215] 09:17:13,125 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/shard-member-3:snapshot-read#256794215] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,125 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.SnapshotOffer from Actor[pekko://cluster-test/system/in-memory-snapshot-store#262639474] 09:17:13,125 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: SnapshotOffer called. 09:17:13,125 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from -1 to 0 09:17:13,125 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Set currentTerm=1, votedFor=null 09:17:13,125 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 09:17:13,125 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#432092899 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 09:17:13,125 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#432092899] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,130 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1894904384] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 1.497 ms 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 09:17:13,132 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1894904384] 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 6.749 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 09:17:13,132 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1894904384] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:13,132 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 1, size: 267 09:17:13,132 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-config_testAddShardReplica, address=null, votingState=VOTING]] 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 1 to 1 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 0 to 1 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 2, size: 235 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 2 to 2 09:17:13,133 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 09:17:13,133 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 551.5 μs 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@37d88423}, purged=MutableUnsignedLongSet{size=0}} 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 1 to 2 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received ReplicatedLogEntry for recovery: index: 3, size: 56 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[pekko://cluster-test/deadLetters] 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Received apply journal entries for recovery, applying to state: 3 to 3 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: Moving last applied index from 2 to 3 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 13.38 ms - Switching actor to Follower - last log index = 3, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 3 09:17:13,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:13,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:13,138 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 09:17:13,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testAddShardReplica is no longer the leader 09:17:13,139 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-877416096] 09:17:13,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-877416096] 09:17:13,139 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:17:13,139 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:17:13,139 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 09:17:13,139 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1036404327] 09:17:13,140 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:13,140 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 09:17:13,736 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:14,66 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-6750405162752676084] but current is [-659586281271436772]. Reason: ActorSystem terminated 09:17:14,68 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-6750405162752676084] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:14,101 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: fakeSnapshot purging log to 2 for term 1 09:17:14,101 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testAddShardReplica, leaderId=member-1-shard-cars-config_testAddShardReplica, leaderPayloadVersion=13] 09:17:14,101 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 09:17:14,102 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:14,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:14,102 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:14,103 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:17:14,103 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:17:14,106 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:14,247 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:14,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - New leader address: pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:14,558 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:14,567 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:14,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-1682807885]] intended for system uid [-6750405162752676084], self uid is [-659586281271436772] 09:17:14,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-1388025810]] intended for system uid [-6750405162752676084], self uid is [-659586281271436772] 09:17:14,887 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-980516173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:15,126 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:16,146 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:17,166 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:17,817 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 09:17:17,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:17,817 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:17,979 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 09:17:17,980 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:17,980 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:18,187 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:17:18,747 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#-1682807885]] intended for system uid [-6750405162752676084], self uid is [-659586281271436772] 09:17:18,837 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 09:17:18,839 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-6750405162752676084] but current is [-659586281271436772]. Reason: Cluster member removed, previous status [Down] 09:17:18,839 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:18,839 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:18,916 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-1388025810]] intended for system uid [-6750405162752676084], self uid is [-659586281271436772] 09:17:19,4 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:19,4 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:19,4 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-6750405162752676084] but current is [-659586281271436772]. Reason: Cluster member removed, previous status [Down] 09:17:19,197 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:19,200 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:19,201 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:19,202 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:17:19,202 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:17:19,204 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6192807203, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:17:19,205 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=17370, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:17:19,206 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:17:19,206 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:17:19,207 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1386778553], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 346.2 μs 09:17:19,208 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 09:17:19,210 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:17:19,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 09:17:19,210 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:17:19,210 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onShutDown 09:17:19,210 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testAddShardReplica 09:17:19,210 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:19,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:19,212 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:19,213 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:19,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:19,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:19,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:19,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:19,237 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:19,238 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 09:17:19,238 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:17:19,238 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onShutDown 09:17:19,239 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testAddShardReplica 09:17:19,239 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:19,239 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:19,240 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:19,240 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 09:17:19,240 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 09:17:19,240 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onShutDown 09:17:19,240 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testAddShardReplica 09:17:19,241 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:19,241 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:19,242 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:19,242 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:20,225 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-659586281271436772] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:20,255 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-6658571468512167435] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:20,261 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:20,262 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:20,265 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:20.267] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:20,279 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 09:17:20,279 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 09:17:20,279 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:17:20,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onShutDown 09:17:20,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: Initiating leader transfer 09:17:20,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4047343c 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4047343c 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Attempting to transfer leadership 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:20,280 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 09:17:20,876 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:20,877 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:20,877 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:20,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:17:20,878 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:17:21,896 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:24,946 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:17:24,947 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:24,947 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:26,722 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:17:26,986 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:17:26,987 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:26,987 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:28,2 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:17:28,5 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:29,275 AM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-config_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 09:17:29,275 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-config_testAddShardReplica, member-3-shard-cars-config_testAddShardReplica] 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 8.996 s 09:17:29,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 09:17:29,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: leader transfer failed - sending PoisonPill 09:17:29,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:29,278 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Leader to IsolatedLeader 09:17:29,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 09:17:29,279 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testAddShardReplica 09:17:29,279 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:29,279 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:29,280 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 09:17:29,280 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 09:17:29,280 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:17:29,280 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onShutDown 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: Initiating leader transfer 09:17:29,281 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testAddShardReplica, leaderId=null, leaderPayloadVersion=13] 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@672f668b 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@672f668b 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Attempting to transfer leadership 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,281 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,285 AM [cluster-test-shard-dispatcher-18] [WARN] Shard - member-1-shard-cars-oper_testAddShardReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 09:17:29,285 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testAddShardReplica, member-2-shard-cars-oper_testAddShardReplica] 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testAddShardReplica: active: false, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 5.059 ms 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: leader transfer failed - sending PoisonPill 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:29,286 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 09:17:29,286 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 09:17:29,286 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testAddShardReplica 09:17:29,287 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:29,287 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:29,288 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:29,288 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:29,291 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:29.293] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:29,320 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:29,329 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7959017844584238652] 09:17:29,334 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:29,334 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:29,334 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:29,335 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:29,335 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:29,347 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,348 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,355 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:29,355 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:29,355 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:29,357 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,361 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 09:17:29,361 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 09:17:29,362 AM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 09:17:29,365 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:29,366 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-807854858] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,367 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:29,367 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,368 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,368 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,368 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,368 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6ced0182} 09:17:29,373 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,374 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,374 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1203863621] 09:17:29,375 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,375 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,376 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,376 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#461843951] 09:17:29,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1898578433 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,376 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,376 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,377 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1045654444 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,377 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,377 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,377 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:29,377 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,377 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,377 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,379 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,379 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#78513652] 09:17:29,379 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1203863621] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,379 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1898578433] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,379 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#461843951] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,379 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1045654444] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,380 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#78513652] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,380 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1025336473] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,381 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,381 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,381 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,381 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,381 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:29,381 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,381 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:29,381 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,381 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,381 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,381 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,381 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,382 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,383 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:29,383 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,384 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,384 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,384 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,384 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,384 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,386 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6578ce8d} 09:17:29,387 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,388 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,388 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,389 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1318162882] 09:17:29,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,389 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1318162882] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,389 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,389 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,389 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#871384816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,390 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#6974510 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-1505688472] 09:17:29,390 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,390 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#339464679 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,390 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-1:snapshot-read#-364985843] 09:17:29,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,390 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,390 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,391 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,390 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,391 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#871384816] 09:17:29,391 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:29,391 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,391 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,391 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,392 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,392 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:29,392 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,392 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,392 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,394 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:29,402 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [1144097098160793129] 09:17:29,402 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:29,405 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:29,405 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:29,405 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:29,405 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:29,418 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,420 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,422 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,430 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#564412605] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,431 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:29,433 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:29,433 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,434 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,435 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,436 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@e0ffb6b} 09:17:29,436 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,436 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,436 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,436 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,437 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,438 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,438 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,438 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,439 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#672177268] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,439 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#610644132] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,439 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,440 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:29,444 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#629511739 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,444 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1664071217 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,444 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#387196700] 09:17:29,445 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#629511739] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,445 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-745017544] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,445 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1664071217] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,445 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#387196700] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,446 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-753468116] 09:17:29,446 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-753468116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,451 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,452 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:29,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,452 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:29,452 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,452 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:29,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,452 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,452 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,452 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,452 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,453 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,453 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,453 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,460 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,463 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,464 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@19f78f10} 09:17:29,464 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,464 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,464 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,465 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,466 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,466 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-814188424] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,466 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#500975850 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,466 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,466 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#500975850] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,467 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#-586608479] 09:17:29,467 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,467 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,467 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-2:snapshot-read#1061713944] 09:17:29,467 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-2005851566 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,468 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,468 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,468 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:29,468 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,468 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,468 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,468 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,477 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:29,480 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,481 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,486 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:29,495 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4658883974327785073] 09:17:29,495 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:29,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:29,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:29,502 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:29,502 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:29,516 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,517 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,522 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,527 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:29,529 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1356801254] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,533 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:29,533 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,533 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,535 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,537 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1900f62b} 09:17:29,538 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,541 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,541 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,541 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,542 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,542 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-128096899] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,543 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:29,543 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,543 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,544 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#23803693 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1699692539] 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1581618025 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#789152066] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1647053685] 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#23803693] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1699692539] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1581618025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,544 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#1647053685] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,546 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:29,546 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,547 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:29,547 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,548 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,548 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#373627987] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,549 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,549 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:17:29,550 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:29,552 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:29,552 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:29,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,553 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,553 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,553 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,555 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:29,555 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,555 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,555 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,555 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:29,556 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,556 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,556 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,556 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,556 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,556 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,556 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,557 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@67e6c2b8} 09:17:29,557 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:17:29,557 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:29,557 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1691420730] 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1575067253] 09:17:29,558 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:29,559 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,559 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1575067253] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,559 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, persistent : true 09:17:29,559 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1691420730] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:29,559 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-1036916714] 09:17:29,559 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1197619244 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,560 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#567282338 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,560 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/shard-member-3:snapshot-read#-601477869] 09:17:29,560 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,560 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with journal batch size 1 09:17:29,560 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,560 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:29,560 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:29,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,560 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:29,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:29,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:17:29,561 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,561 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,561 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,561 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,561 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,561 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,561 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:17:29,561 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,561 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,561 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:17:29,561 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,561 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:29,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,722 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,723 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,723 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:17:29,723 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:17:29,723 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:17:29,723 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:17:29,723 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:17:29,723 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:17:29,771 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:29,771 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Received ElectionTimeout - switching to Candidate 09:17:29,771 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 09:17:29,771 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:29,772 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:17:29,772 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:29,772 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:17:29,772 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is no longer the leader 09:17:29,772 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:17:29,772 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:17:30,82 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,82 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,82 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:17:30,101 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,101 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,101 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:17:30,141 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,141 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,141 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,141 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:17:30,141 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,141 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:17:30,352 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:30,353 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,353 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,353 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,354 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,354 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,355 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,355 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,355 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,356 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,356 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,362 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,363 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,433 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,433 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,433 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:17:30,433 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,433 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,435 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,436 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,436 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,436 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards] 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:17:30,437 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:17:30,437 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,438 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:17:30,438 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,438 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,439 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,439 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,439 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,439 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,440 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,440 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,441 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,482 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,482 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:17:30,482 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:17:30,482 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,482 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,485 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,486 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:30,486 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards] 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards] 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,487 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,487 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:17:30,487 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:17:30,488 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,488 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:17:30,488 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:17:30,488 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:17:30,488 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,488 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,489 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,489 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,489 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:17:30,489 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,489 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,489 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,490 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,490 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,490 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,490 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,489 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,491 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,491 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:17:30,491 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,491 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,491 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,518 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:17:30,551 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Received ElectionTimeout 09:17:30,551 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,551 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:17:30,552 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,552 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:30,555 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Cluster state: [] 09:17:30,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:30,555 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:30,555 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=null 09:17:30,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:30,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:30,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:30,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:30,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards] 09:17:30,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:17:30,557 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:17:30,557 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:17:30,557 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:17:30,557 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,557 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:17:30,557 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,559 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,559 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,560 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:17:30,560 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,560 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,561 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderPayloadVersion=13] 09:17:30,561 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:17:30,561 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:17:30,561 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:17:30,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-807854858] 09:17:30,561 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:30,561 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: -1, nextIndex: 0 09:17:30,561 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] 09:17:30,561 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:30,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] 09:17:30,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#564412605] 09:17:30,563 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:30,563 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#672177268] 09:17:30,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#672177268] 09:17:30,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1356801254] 09:17:30,563 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:30,564 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-128096899] 09:17:30,564 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-128096899] 09:17:30,566 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 09:17:30,566 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 09:17:30,567 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 09:17:30,567 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:30,568 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,569 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards)] 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-config_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards)] 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,569 AM [cluster-test-shard-dispatcher-32] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards)] 09:17:30,569 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: getLeaderAddress leaderId = member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,569 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards)] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=45e61b60-cf9a-4679-b2c8-4ec8bff30339}, logIndex: 0, isSendImmediate: true 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=a9f295eb-30fe-4a0d-8515-2820066f0676}, logIndex: 0, isSendImmediate: true 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 09:17:30,572 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,573 AM [cluster-test-shard-dispatcher-35] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,573 AM [cluster-test-shard-dispatcher-35] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:30,573 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=72639586-e6a9-48e8-bbb3-bf20f9a69c99}, logIndex: 0, isSendImmediate: true 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:30,573 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,573 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,574 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,574 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]], state: Idle 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: New server configuration : [ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=edaaadf0-1a4d-4f88-bbd4-befb4481a202}, logIndex: 0, isSendImmediate: true 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Setting last applied to 0 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,575 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,576 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] 09:17:30,576 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:30,576 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,576 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,579 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,580 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,580 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:17:30,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:17:30,580 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:17:30,580 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:17:30,580 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:17:30,580 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:17:30,580 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-807854858] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING]] 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,581 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,581 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,581 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] 09:17:30,581 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#76408609] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]]]] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Number of entries to be appended = 1 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, votingState=NON_VOTING]] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Log size is now 1 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Commit index set to 0 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Setting last applied to 0 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Moving last applied index from -1 to 0 09:17:30,582 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]] 09:17:30,582 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,582 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,582 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,582 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,583 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,583 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,583 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,583 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,583 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,583 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,583 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]]] 09:17:30,583 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Persisting ApplyJournalEntries with index=0 09:17:30,583 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,583 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,583 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-807854858] 09:17:30,583 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:17:30,583 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,583 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:30,583 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,583 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1203863621] 09:17:30,583 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1203863621] 09:17:30,583 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,584 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards updated: matchIndex: 0, nextIndex: 1 09:17:30,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1025336473] 09:17:30,584 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:30,584 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:30,584 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1318162882] 09:17:30,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1318162882] 09:17:30,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1025336473] 09:17:30,584 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:17:30,584 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#871384816] 09:17:30,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#871384816] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#564412605] 09:17:30,585 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:30,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#672177268] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#672177268] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#564412605] 09:17:30,585 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:17:30,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#610644132] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#610644132] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-745017544] 09:17:30,585 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:30,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-814188424] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-814188424] 09:17:30,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-745017544] 09:17:30,585 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:17:30,585 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-177518570] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-177518570] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1356801254] 09:17:30,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:30,586 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-128096899] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-128096899] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1356801254] 09:17:30,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:17:30,586 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#789152066] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#789152066] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#373627987] 09:17:30,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:30,586 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1691420730] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1691420730] 09:17:30,586 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#373627987] 09:17:30,586 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:17:30,587 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1575067253] 09:17:30,587 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1575067253] 09:17:30,587 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,587 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,587 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,588 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:30,588 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,588 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,589 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,589 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:30,589 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:30,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:30,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:30,593 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:30,593 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:30,594 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:30,594 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:30,599 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:30,619 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,620 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,620 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,620 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:17:30,620 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,620 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,621 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:30,621 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:30,621 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:30,621 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:30,622 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:30,622 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,622 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,622 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:30,622 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,622 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:30,623 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:30,623 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:30,623 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:30,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:30,625 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:30,625 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:31,372 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:31,601 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-4658883974327785073] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:31,642 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [1144097098160793129] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:31,643 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:31,643 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:31,647 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:31.649] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:31,664 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:31,665 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:31,665 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:31,665 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@33282870 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@33282870 09:17:31,665 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cc094cc 09:17:31,665 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cc094cc 09:17:31,665 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:17:31,665 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 09:17:31,666 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:31,666 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:32,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:17:32,32 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 366.6 ms 09:17:32,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:17:32,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 09:17:32,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 09:17:32,33 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:17:32,81 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:17:32,82 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 416.4 ms 09:17:32,82 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:17:32,82 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 09:17:32,82 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 09:17:32,83 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:17:32,83 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:32,83 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:32,84 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onShutDown 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Initiating leader transfer 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@762ab071 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@762ab071 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:17:32,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 09:17:32,84 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c526cbe 09:17:32,85 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c526cbe 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Attempting to transfer leadership 09:17:32,85 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 09:17:32,85 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, leaderId=null, leaderPayloadVersion=13] 09:17:32,402 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:17:32,402 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 317.7 ms 09:17:32,402 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:17:32,402 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 09:17:32,402 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 09:17:32,403 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:17:32,472 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:17:32,472 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 387.5 ms 09:17:32,472 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:17:32,472 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer failed - sending PoisonPill 09:17:32,472 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Leader): Leadership transfer expired 09:17:32,473 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:17:32,473 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:32,474 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:32,475 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:32,475 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:32,479 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:32.481] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:32,505 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:17:32,512 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [5533030566100762319] 09:17:32,513 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:32,516 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:32,516 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:32,525 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,526 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:32,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:32,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:32,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:32,528 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:32,529 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 09:17:32,533 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,533 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:32,537 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:32,537 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 09:17:32,538 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,538 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:17:32,540 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 09:17:32,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6298e029} 09:17:32,541 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 09:17:32,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 09:17:32,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,542 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:17:32,542 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:17:32,542 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1013882186] 09:17:32,542 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-default-config_testGetPrefixShardRole, persistent : true 09:17:32,542 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1013882186] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,546 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,546 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1772616575 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 09:17:32,547 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1772616575] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,547 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#902709987] 09:17:32,548 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/shard-member-1:snapshot-read#902709987] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,548 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-default-config_testGetPrefixShardRole with journal batch size 1 09:17:32,550 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 09:17:32,552 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1045967606] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,553 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:32,559 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,559 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:32,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 09:17:32,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 09:17:32,560 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:32,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-config_testGetPrefixShardRole 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 09:17:32,560 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:32,560 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,561 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,561 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,561 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-config_testGetPrefixShardRole is no longer the leader 09:17:32,561 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 09:17:32,561 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Received ElectionTimeout 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:32,563 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 09:17:32,563 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 09:17:32,563 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 09:17:32,563 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-config_testGetPrefixShardRole, leaderId=member-1-shard-default-config_testGetPrefixShardRole, leaderPayloadVersion=13] 09:17:32,564 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 09:17:32,564 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 09:17:32,564 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3dccd33a} 09:17:32,564 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 09:17:32,565 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,565 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,565 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,565 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,565 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#759219300] 09:17:32,565 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-default-oper_testGetPrefixShardRole, persistent : true 09:17:32,566 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#759219300] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,566 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1230105436 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,566 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#1177131251] 09:17:32,566 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-1230105436] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,566 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-default-oper_testGetPrefixShardRole with journal batch size 1 09:17:32,566 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/shard-member-1:snapshot-read#1177131251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:32,567 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 09:17:32,567 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower): Received TimeoutNow - switching to Candidate 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Election: Candidate has following voting peers: [] 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Set currentTerm=1, votedFor=member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,567 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard member-1-shard-default-oper_testGetPrefixShardRole is no longer the leader 09:17:32,567 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 09:17:32,567 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Received ElectionTimeout 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Leader): Election: Leader has following peers: [] 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 09:17:32,568 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 09:17:32,568 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-oper_testGetPrefixShardRole, leaderId=member-1-shard-default-oper_testGetPrefixShardRole, leaderPayloadVersion=13] 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 09:17:32,568 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 09:17:32,615 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] 09:17:32,616 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 09:17:32,617 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1013882186] 09:17:32,617 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 09:17:32,618 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 09:17:32,618 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:17:32,618 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onShutDown 09:17:32,618 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@330b3af3 09:17:32,618 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@330b3af3 09:17:32,620 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-default-config_testGetPrefixShardRole 09:17:32,620 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:32,620 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:32,621 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:17:32,622 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onShutDown 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@f16a7be 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@f16a7be 09:17:32,622 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1703463898] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-1-shard-default-oper_testGetPrefixShardRole 09:17:32,622 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:32,622 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:32,623 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:32,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:32,626 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:32,645 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:32,656 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5555541954995067747] 09:17:32,657 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:32,660 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:32,660 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:32,664 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:32,664 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:32,673 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,675 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:32,676 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:32,676 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:32,676 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,679 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:32,684 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,687 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:32,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,687 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,688 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,688 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,688 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,690 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,691 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6ec18076} 09:17:32,691 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,692 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,693 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,696 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,697 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-299961618] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,698 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:32,698 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,698 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,699 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1211770437] 09:17:32,699 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1211770437] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,699 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,699 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2003520473] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,699 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2003520473 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,700 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:32,700 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,701 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,701 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,701 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#929685415] 09:17:32,701 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,701 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,701 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2013804012 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,701 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1016642519] 09:17:32,702 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#929685415] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,702 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2013804012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,702 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1016642519] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:32,705 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:32,706 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,706 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,706 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,706 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:32,706 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,707 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,708 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,708 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@767726d} 09:17:32,708 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,709 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,709 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,712 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,713 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,713 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#-2040466260] 09:17:32,713 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,713 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#-2040466260] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,714 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#825742946 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,714 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,714 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,714 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,715 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,716 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,717 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,717 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,717 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#363279530] 09:17:32,718 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:32,718 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,718 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,718 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,719 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,719 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,719 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-1:snapshot-read#1406356252] 09:17:32,719 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-918141780 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,722 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:32,722 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,723 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,723 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,723 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,732 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:32,743 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [4783679391703157221] 09:17:32,744 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:32,748 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:32,748 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:32,748 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:32,748 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:32,760 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,765 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,767 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,772 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#934814852] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,775 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:32,781 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:32,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,781 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,783 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,784 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,784 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@366c370} 09:17:32,785 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,785 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,785 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,786 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,789 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,790 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:32,792 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,793 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,793 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-749620057] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,793 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1043246089] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,793 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#282107714] 09:17:32,793 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,793 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,794 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#547826380 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#282107714] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1490815730] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#547826380] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,796 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,796 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,796 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,797 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:32,799 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:32,799 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,800 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:32,801 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,801 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,801 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,801 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,801 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,802 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-553157620] 09:17:32,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-706371263 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-706371263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-553157620] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,803 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:32,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,803 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,803 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,803 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,805 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,805 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:32,805 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,805 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,805 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,806 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,806 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,806 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,806 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@3ef9b101} 09:17:32,807 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,807 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,808 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,808 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,808 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1916502777] 09:17:32,808 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,809 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,809 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,809 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#91472695] 09:17:32,809 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,809 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,810 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,810 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,810 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1916502777] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,811 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#91472695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,811 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#110322430 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,811 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#766784642] 09:17:32,811 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-983311335 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-2:snapshot-read#-1468495771] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,812 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,812 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,812 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,812 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,812 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,812 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,812 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:32,819 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [7203548870383319149] 09:17:32,819 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:32,821 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:32,821 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:32,823 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:32,823 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:32,830 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,831 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,835 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,853 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-239347632] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,856 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:32,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:32,860 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:32,864 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:32,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,865 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,865 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,865 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,866 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7d65e6fd} 09:17:32,866 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,867 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,867 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,867 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1303632475] 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-340941519] 09:17:32,868 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,869 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:32,869 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1267213762 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,870 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1303632475] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,870 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#666311484 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,870 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-340941519] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,870 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1267213762] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,870 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#666311484] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,871 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,871 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-696402984] 09:17:32,872 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-696402984] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,872 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,873 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,876 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,877 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1341411166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:32,877 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,880 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,880 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,881 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,881 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:32,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,881 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,881 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,881 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#278217242] 09:17:32,882 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#278217242] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,882 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,882 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,882 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:32,882 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,883 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,884 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,885 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,885 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,885 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,885 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,885 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:32,885 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,885 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,885 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,887 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:32,887 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@389343a} 09:17:32,888 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#730627033] 09:17:32,889 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,890 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,890 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,890 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#843025601] 09:17:32,890 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#730627033] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,890 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,890 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, persistent : true 09:17:32,891 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1664011413 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,891 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#843025601] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:32,891 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,892 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-854095629] 09:17:32,892 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-44605325 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,892 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,892 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,892 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,893 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,893 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,893 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,893 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:32,893 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with journal batch size 1 09:17:32,893 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/shard-member-3:snapshot-read#-1130481977] 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=1, votedFor=null 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Received ReplicatedLogEntry for recovery: index: 0, size: 567 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING]] 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:32,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:32,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:32,894 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:32,895 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,895 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:17:32,895 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:32,895 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:17:33,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:33,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:33,682 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,682 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,683 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:33,690 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,690 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:33,691 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,691 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,692 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] 09:17:33,692 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:33,692 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,692 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,693 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:17:33,693 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,693 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:17:33,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:17:33,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:17:33,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:33,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:17:33,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:17:33,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:33,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:33,694 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,694 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 09:17:33,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:33,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 09:17:33,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 09:17:33,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:33,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 09:17:33,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 09:17:33,694 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:33,695 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]], state: Idle 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:17:33,695 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:33,695 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]], state: Idle 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:33,695 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Sending local ElectionTimeout to start leader election 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Received TimeoutNow - switching to Candidate 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:17:33,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:17:33,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Sending RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is no longer the leader 09:17:33,696 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:17:33,696 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:17:33,697 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,697 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,697 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Cluster state: [] 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,698 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,698 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Candidate in requestVote:RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=null 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): In requestVote: RequestVote [term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1] - currentTerm: 2, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,699 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): requestVote returning: RequestVoteReply [term=2, voteGranted=true] 09:17:33,699 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:17:33,699 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:17:33,699 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:33,699 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader elected 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=3f894f52-d412-4e2a-92ee-9fd6a7b1c5dd}, logIndex: 1, isSendImmediate: true 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=d484f93b-c9f6-4005-8390-b33a5e502793}, logIndex: 1, isSendImmediate: true 09:17:33,700 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,700 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,700 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:17:33,701 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): RaftRPC message received RequestVoteReply [term=2, voteGranted=true], my term is 2 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=2, voteGranted=true], current voteCount: 1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Election: Leader has following peers: [member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader elected 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New leader member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader elected 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: New server configuration : [ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]] 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,701 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=caf1f166-cb80-4c11-a8eb-a14621ed9f78}, logIndex: 1, isSendImmediate: true 09:17:33,701 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: ServerOperationContextIdentifier{value=bbd73b68-253c-481d-a940-8713de2a2775}, logIndex: 1, isSendImmediate: true 09:17:33,701 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 09:17:33,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,702 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:17:33,702 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,702 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,702 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,702 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,702 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:33,703 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:17:33,703 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:17:33,703 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] 09:17:33,703 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:17:33,703 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:33,703 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,703 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,704 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] 09:17:33,704 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,704 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,704 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:17:33,704 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,704 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persist data SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Replicate message: identifier: null, logIndex: 2, isSendImmediate: true 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderPayloadVersion=13] 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,705 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:17:33,705 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,705 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:33,705 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#929685415] 09:17:33,705 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#929685415] 09:17:33,706 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-299961618] 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:33,706 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,706 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,706 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,706 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 0, nextIndex: 1 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] 09:17:33,707 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-299961618] 09:17:33,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:17:33,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,707 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#363279530] 09:17:33,708 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#363279530] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,708 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#934814852] 09:17:33,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,708 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1043246089] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,709 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1043246089] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,709 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,710 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,710 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,710 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,710 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,710 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,710 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#934814852] 09:17:33,711 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,711 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:17:33,711 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1490815730] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1490815730] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-749620057] 09:17:33,711 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:33,711 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1916502777] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1916502777] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-749620057] 09:17:33,711 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:17:33,711 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#91472695] 09:17:33,711 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#91472695] 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]], SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Number of entries to be appended = 2 09:17:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,711 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,711 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 09:17:33,711 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-239347632] 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:33,712 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1303632475] 09:17:33,712 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, address=null, votingState=NON_VOTING]] 09:17:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1303632475] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,712 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,712 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Log size is now 3 09:17:33,712 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=2, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, logLastIndex=2, logLastTerm=2, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:33,712 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 2, lastIndex: 2 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,712 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=1, term=2, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]]] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 09:17:33,712 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: persist complete SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 09:17:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-239347632] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,712 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,712 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-340941519] 09:17:33,712 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,712 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-340941519] 09:17:33,713 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,713 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,713 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 2, lastIndex: 2 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 2, lastIndex: 2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 0 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Setting last applied to 2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader updated: matchIndex: 2, nextIndex: 3 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:33,713 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:17:33,713 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=2, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 2, lastIndex: 2 09:17:33,713 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:17:33,713 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,713 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:17:33,713 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,714 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,714 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:17:33,714 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:17:33,714 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,713 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1341411166] 09:17:33,713 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader] 09:17:33,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,714 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:17:33,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:17:33,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,715 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:17:33,715 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:17:33,715 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,715 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,715 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,715 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,716 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#730627033] 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#730627033] 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1341411166] 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,716 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,716 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:17:33,716 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Commit index set to 2 09:17:33,716 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): -1, lastIndex(): 2 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 0 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from -1 to 0 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 1 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 0 to 1 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 1 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]] 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Setting last applied to 2 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Moving last applied index from 1 to 2 09:17:33,717 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,717 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Persisting ApplyJournalEntries with index=2 09:17:33,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#843025601] 09:17:33,717 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#843025601] 09:17:33,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-406076536] 09:17:33,718 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:33,718 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1302356020] 09:17:33,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-299961618] 09:17:33,719 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:33,719 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] 09:17:33,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-117331420] 09:17:33,720 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:33,720 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,720 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,720 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,720 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,720 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,721 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,721 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:17:33,721 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:33,721 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:33,721 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:33,721 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,721 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,721 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:33,722 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,722 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,722 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,722 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,722 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:33,722 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:33,723 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:33,723 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:33,727 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:33,727 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:33,742 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:33,742 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,742 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,742 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:17:33,742 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,743 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,743 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,743 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,743 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:33,743 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:33,743 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:33,744 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,744 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,744 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,744 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:33,744 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,744 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,744 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,744 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:33,744 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:33,746 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:33,746 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:33,749 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:33,749 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:33,750 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:33.753] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:33,763 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:33,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:33,764 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:33,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 09:17:33,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 09:17:33,764 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4265fce5 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4265fce5 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@dcf5580 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@dcf5580 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:33,765 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:34,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:17:34,351 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 587.0 ms 09:17:34,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:17:34,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 09:17:34,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 09:17:34,352 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:17:34,352 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 587.7 ms 09:17:34,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:17:34,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 09:17:34,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 09:17:34,353 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:34,353 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:34,354 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onShutDown 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Initiating leader transfer 09:17:34,354 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37475245 09:17:34,354 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, leaderId=null, leaderPayloadVersion=13] 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@31a6e564 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37475245 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@31a6e564 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:17:34,354 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:17:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 09:17:34,355 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Attempting to transfer leadership 09:17:34,355 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:34,355 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:34,355 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:34,355 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:34,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:17:34,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:17:34,670 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 315.9 ms 09:17:34,670 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 315.9 ms 09:17:34,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:17:34,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:17:34,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 09:17:34,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer failed - sending PoisonPill 09:17:34,670 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 09:17:34,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Leader): Leadership transfer expired 09:17:34,671 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,671 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:17:34,672 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:34,672 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:34,673 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:34,673 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:34,700 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:35,693 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:35.695] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:35,711 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:17:35,720 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [6477087639953069856] 09:17:35,720 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:35,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:35,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:35,729 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:35,729 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:35,738 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:35,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:35,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:35,740 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,743 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1324673078] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,747 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:35,748 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:35,748 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,749 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,749 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,749 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,749 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,752 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,757 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@54e1c9d9} 09:17:35,757 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,757 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,757 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,757 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1427670914] 09:17:35,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,759 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,759 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,759 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,759 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,759 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1427670914] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,760 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1917124946 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,761 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,761 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#492330502] 09:17:35,761 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-596924663] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,762 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,763 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:35,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,764 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,764 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-596924663 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,764 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,764 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,764 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,765 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-511645715] 09:17:35,765 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1917124946] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,765 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#492330502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,765 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1347590395] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,765 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#-511645715] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,769 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,776 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:35,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,776 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:35,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,777 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,777 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,777 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,777 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,777 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,777 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,777 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,777 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,777 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,778 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,778 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,778 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7ee185ce} 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-553586520] 09:17:35,780 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,780 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,781 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,781 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,781 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,781 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#854641675 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,781 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1002836408 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,781 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#580951330] 09:17:35,781 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-1:snapshot-read#689135118] 09:17:35,782 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-553586520] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,782 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-442942377] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,782 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,782 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,782 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-442942377] 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,784 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,784 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,785 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,785 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:35,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,785 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:35,785 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,785 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,785 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,785 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,785 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,785 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,793 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:35,799 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-8585976410723450945] 09:17:35,799 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:35,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:35,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:35,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:35,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:35,813 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,814 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,816 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,826 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:35,827 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#931689830] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,829 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:35,829 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,829 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,831 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,832 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@2da13c3e} 09:17:35,832 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,833 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,833 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,833 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,834 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,837 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,842 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,844 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-113932401] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,844 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,845 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2112906627] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,845 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-209380443 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-209380443] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,845 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1408820772] 09:17:35,846 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1408820772] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,846 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,846 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1314807731 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,847 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:35,848 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1945371354] 09:17:35,852 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,853 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1314807731] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,853 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1092789772] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,853 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#-1945371354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,857 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,857 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,857 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,857 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,858 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,858 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,858 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,858 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:35,858 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,858 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,858 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,858 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,858 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,858 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@38fa5553} 09:17:35,859 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,859 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,859 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,859 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1756130532] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,860 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Shard created : member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,860 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,860 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2083399236 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,860 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#289821619] 09:17:35,860 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-2083399236] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,861 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Shard created : member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,861 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:35,861 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,861 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,861 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,861 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,861 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#571530225 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,862 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-2:snapshot-read#546064208] 09:17:35,863 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:35,864 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,864 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,865 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,865 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,865 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,865 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,865 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:35,865 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,865 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,865 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,865 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,874 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:35,879 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,879 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,880 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,880 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,880 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,881 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:35,894 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [8722906258665201761] 09:17:35,894 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:35,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:35,913 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,914 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,921 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,926 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-420706075] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,927 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:35,930 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:35,930 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,930 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,931 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,938 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,939 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,940 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@235dbbbe} 09:17:35,940 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,940 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,940 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:35,943 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,943 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:35,944 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,945 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,945 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,945 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,945 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,946 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:35,948 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,948 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,948 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,948 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,948 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-265496034 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1305692089 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#652970025] 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#200349095] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1862937669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1432188093] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-265496034] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,950 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1305692089] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,951 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#652970025] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,953 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-326233269] 09:17:35,953 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-326233269] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,955 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:35,955 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,955 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:35,955 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,955 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,956 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,956 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,956 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,957 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,958 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,959 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1495da54} 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 603 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING]] 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,959 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,959 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,959 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-391765433] 09:17:35,960 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-240905486] 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-391765433] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,960 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Shard created : member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,960 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, persistent : true 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#108212067 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-240905486] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#550360769 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,960 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#-785905628] 09:17:35,961 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/shard-member-3:snapshot-read#23517114] 09:17:35,961 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,961 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with journal batch size 1 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=1, votedFor=null 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 591 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Received ReplicatedLogEntry for recovery: index: 0, size: 579 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: true, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:35,962 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,962 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:17:35,962 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:35,962 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:17:35,962 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,962 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:35,962 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:35,962 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:17:36,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:36,98 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:17:36,98 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:36,118 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:17:36,118 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:36,138 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:17:36,138 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Received ElectionTimeout - switching to Candidate 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=2, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is no longer the leader 09:17:36,148 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:17:36,148 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:17:36,458 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,458 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,458 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:17:36,498 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,498 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,498 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:17:36,508 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,508 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,508 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:17:36,557 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,557 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=3, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,557 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:17:36,739 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:36,739 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,740 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,740 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,741 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,741 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,747 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,748 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,828 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:17:36,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,828 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,830 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,830 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,832 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,833 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,833 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,833 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:17:36,833 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,833 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,834 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,834 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,835 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,835 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,835 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,835 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,835 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,836 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:17:36,836 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,837 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,837 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,837 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:17:36,837 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,839 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,840 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,840 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,848 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,848 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,848 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:17:36,848 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,848 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,849 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,849 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,850 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,850 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,851 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,851 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,852 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,852 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,852 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,853 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,853 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,853 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,853 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,853 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,854 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,854 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,854 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:17:36,854 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,855 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,858 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:17:36,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,859 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:17:36,860 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,860 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:17:36,860 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:17:36,860 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,861 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,861 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,862 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,862 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,862 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,862 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,862 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,862 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,862 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,863 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,864 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:17:36,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,864 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,864 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 09:17:36,864 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] is present: existingEntryTerm: 4 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,865 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Received ElectionTimeout 09:17:36,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,897 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:17:36,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] to peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,898 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,899 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,899 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Cluster state: [] 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] with higher term appears reachable, updating term. 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1]" message is greater than follower's term 1 - updating term 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=null 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 1 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Set currentTerm=4, votedFor=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:36,901 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:17:36,902 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:17:36,902 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,902 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Replicate message: identifier: null, logIndex: 1, isSendImmediate: true 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,902 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,902 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,903 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,903 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,903 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:17:36,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,904 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderPayloadVersion=13] 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,904 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 0, nextIndex: 1 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,904 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,904 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:17:36,904 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,904 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,905 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,905 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,905 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 1 does not match the current term 4 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 0 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Setting last applied to 1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,906 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader): Received ApplyState [identifier=null, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]] - lastApplied: 1, lastIndex: 1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Election: Leader has following peers: [member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:17:36,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:17:36,907 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:17:36,907 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:17:36,907 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] 09:17:36,907 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 09:17:36,907 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1324673078] 09:17:36,907 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:36,907 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:17:36,907 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=0, prevLogTerm=1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21]]] 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21] is present: existingEntryTerm: 4 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 2 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,907 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,907 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,907 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Commit index set to 1 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 0 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from -1 to 0 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Setting last applied to 1 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Moving last applied index from 0 to 1 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@674b6b21 09:17:36,908 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persisting ApplyJournalEntries with index=1 09:17:36,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 1, nextIndex: 2 09:17:36,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:17:36,908 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:17:36,908 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,908 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,908 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:17:36,908 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:17:36,909 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:17:36,909 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:36,909 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:36,909 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:17:36,909 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:36,909 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:17:36,910 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]], state: Idle 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,911 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,911 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=28a1c23a-10b1-4ae2-af2c-7f218e3639f5}, logIndex: 2, isSendImmediate: true 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=8afe8e90-4e7a-49e6-96f9-44960a2af7a4}, logIndex: 2, isSendImmediate: true 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]], state: Idle 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Config 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: New server configuration : [ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]] 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=4f7e2810-d8c3-4516-9df4-f9b95df2bee2}, logIndex: 2, isSendImmediate: true 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Config 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=94f5a873-9081-4f16-a157-5ed7877fcb2c}, logIndex: 2, isSendImmediate: true 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Sending AppendEntries to follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:36,912 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - onSuccess for shard cars, type Operational 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard people, type Operational 09:17:36,912 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:17:36,912 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1324673078] 09:17:36,913 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:36,913 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,913 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1324673078] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,913 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,913 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,914 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,914 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,914 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,914 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:17:36,914 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1427670914] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,914 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1427670914] 09:17:36,915 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,915 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,915 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Number of entries to be appended = 1 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Updated server config: isVoting: false, peers: [PeerInfo [id=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING], PeerInfo [id=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, votingState=NON_VOTING], PeerInfo [id=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, address=null, votingState=VOTING]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Log size is now 3 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,915 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=true], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, isVoting=false]]]] 09:17:36,916 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1347590395] 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:17:36,916 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,916 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,916 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-553586520] 09:17:36,916 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-553586520] 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,916 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1347590395] 09:17:36,917 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:17:36,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,917 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-442942377] 09:17:36,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-442942377] 09:17:36,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,917 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,917 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,917 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown updated: matchIndex: 2, nextIndex: 3 09:17:36,917 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:36,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#931689830] 09:17:36,917 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:36,917 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-113932401] 09:17:36,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-113932401] 09:17:36,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#931689830] 09:17:36,917 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2112906627] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2112906627] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1092789772] 09:17:36,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#185406664] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#185406664] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1092789772] 09:17:36,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1756130532] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1756130532] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-420706075] 09:17:36,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#200349095] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#200349095] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-420706075] 09:17:36,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1862937669] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-1862937669] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1432188093] 09:17:36,918 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:17:36,918 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-391765433] 09:17:36,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-391765433] 09:17:36,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1432188093] 09:17:36,919 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:17:36,919 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-240905486] 09:17:36,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-240905486] 09:17:36,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1324673078] 09:17:36,919 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:36,919 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1354394045] 09:17:36,920 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:36,920 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,920 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,920 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:17:36,920 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,920 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,920 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,920 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,921 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:36,921 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:36,921 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:36,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,921 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,921 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:36,921 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,922 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,922 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,922 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,922 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:36,922 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:36,922 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:36,923 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:36,926 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:36,926 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:36,926 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:36,926 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:36,928 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:36,939 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:36,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,940 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:17:36,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,941 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,941 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - Stopping Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,941 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,941 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:36,941 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:36,941 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:36,942 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,942 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,942 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:36,942 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,942 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:36,942 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,942 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:36,942 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:36,942 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:36,945 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:36,945 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:37,528 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 09:17:37,528 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorServerConfigurationSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:37,528 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:37,528 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorServerConfigurationSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:17:37,759 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:37,938 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [8722906258665201761] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:37,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-8585976410723450945] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:37,959 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:37,959 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:37,961 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:37.963] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:37,969 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 09:17:37,969 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@231b84fb 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17e07c1f 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@17e07c1f 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@231b84fb 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:17:37,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 09:17:37,969 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 09:17:37,970 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:37,970 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:37,970 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:37,970 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:37,970 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 09:17:38,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:17:38,428 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 459.0 ms 09:17:38,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:17:38,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 09:17:38,428 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 09:17:38,429 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 488.4 ms 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 09:17:38,458 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,459 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:38,459 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:38,459 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:17:38,459 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onShutDown 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Initiating leader transfer 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29edd63c 09:17:38,460 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29edd63c 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:38,460 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, leaderId=null, leaderPayloadVersion=13] 09:17:38,460 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a982944 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a982944 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Attempting to transfer leadership 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:38,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:17:38,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:17:38,768 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 308.4 ms 09:17:38,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:17:38,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 09:17:38,768 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 09:17:38,769 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,818 AM [cluster-test-shard-dispatcher-34] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:17:38,818 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 357.9 ms 09:17:38,818 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:17:38,818 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer failed - sending PoisonPill 09:17:38,818 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Leader): Leadership transfer expired 09:17:38,819 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:17:38,819 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:38,819 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:38,820 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:38,820 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:38,822 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:38.825] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:38,841 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:17:38,851 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [192629849948048061] 09:17:38,851 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:38,856 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:38,856 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:38,856 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:38,856 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:38,867 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,868 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:38,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:38,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:38,870 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:17:38,873 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:38,874 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#22260378] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,875 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,878 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:38,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@1c5f1172} 09:17:38,879 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:17:38,879 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:38,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#1606763537] 09:17:38,880 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,880 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-1-shard-people-config_testRemoveShardLeaderReplica, persistent : true 09:17:38,880 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 09:17:38,880 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#1606763537] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,880 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-2136361197 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-2136361197] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#1711061049] 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-352432952] 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#1711061049] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,881 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-352432952] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,881 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:38,882 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:17:38,882 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,883 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-471063110 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,883 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,883 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-471063110] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,883 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:38,883 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-1-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,883 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-369542408] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,883 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,884 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 09:17:38,884 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,884 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:38,884 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:17:38,884 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:38,884 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 09:17:38,884 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:38,885 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:17:38,886 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:38,886 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,887 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,887 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:38,887 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,887 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,887 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:38,888 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@77e30ae0} 09:17:38,888 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:17:38,888 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:38,888 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#2067387737] 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,889 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-159566330] 09:17:38,890 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 09:17:38,890 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,890 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Shard created : member-1-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 09:17:38,891 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1034931651 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,891 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-1289850600] 09:17:38,891 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#2067387737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,891 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-159566330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,892 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#952242957 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:38,892 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:38,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:17:38,892 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,892 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:17:38,894 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/shard-member-1:snapshot-read#-548008305] 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:38,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:38,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:17:38,895 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,895 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,895 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:17:38,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:38,908 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4645474035699533924] 09:17:38,909 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:38,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:38,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:38,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:38,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:38,919 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,921 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,922 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:17:38,934 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:38,934 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1009444241] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,936 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:38,938 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:17:38,938 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,940 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:38,941 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@6b840203} 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:38,942 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,947 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#1989569446] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,948 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:38,948 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,948 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 09:17:38,948 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:38,949 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-300214208 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,949 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-300214208] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,949 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-839789206] 09:17:38,949 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-839789206] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,950 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,950 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:38,951 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-people-config_testRemoveShardLeaderReplica, persistent : true 09:17:38,951 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:38,951 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,951 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,951 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:17:38,951 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,952 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@7b463a32} 09:17:38,952 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:38,952 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,952 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,952 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Shard created : member-2-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 09:17:38,952 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,953 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-2-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 09:17:38,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1361200669 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,955 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#1238644432] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#141558282] 09:17:38,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1895112328 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#745095522] 09:17:38,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#1896205472] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#1653256267] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1361200669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,956 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#141558282] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:38,960 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#530535738 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:38,961 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,961 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:38,961 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:38,961 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,961 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:17:38,961 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,961 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,961 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:38,961 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:17:38,964 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/shard-member-2:snapshot-read#-1689223703] 09:17:38,964 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:38,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:38,965 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:38,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:38,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:38,965 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:38,972 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:38,972 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:17:38,972 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:38,972 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:17:38,972 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,972 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:38,972 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:17:38,973 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:17:38,975 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:38,978 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:38,982 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:38,983 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:38,994 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3543878942082743549] 09:17:38,995 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:38,997 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:38,997 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:39,2 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:39,2 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:39,6 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:39,7 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:39,9 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:17:39,14 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:39,18 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1096350183] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,23 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:39,24 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,24 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,27 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:39,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@111b4a51} 09:17:39,28 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:17:39,29 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:39,30 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:39,31 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:39,31 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,32 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:17:39,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,34 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-config_testRemoveShardLeaderReplica, persistent : true 09:17:39,34 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-2064519954] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,35 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#1010429983 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,35 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#68553401] 09:17:39,35 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#1010429983] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,35 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:39,36 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:39,36 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:39,36 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,36 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:39,36 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:39,37 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Shard created : member-3-shard-people-config_testRemoveShardLeaderReplica, persistent : true 09:17:39,37 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,37 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-1637537116 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,37 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1530034072] 09:17:39,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:39,38 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:17:39,38 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#68553401] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,38 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#317403160] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,38 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1267145655] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,38 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#-1637537116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,38 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#1530034072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,38 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,38 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:39,39 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:39,39 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:39,40 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:17:39,40 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:39,42 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Starting recovery for member-3-shard-people-config_testRemoveShardLeaderReplica with journal batch size 1 09:17:39,43 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@658764da} 09:17:39,43 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:17:39,43 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:39,43 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:39,43 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,44 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,44 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-cars-oper_testRemoveShardLeaderReplica, persistent : true 09:17:39,44 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:39,44 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Shard created : member-3-shard-people-oper_testRemoveShardLeaderReplica, persistent : true 09:17:39,45 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testRemoveShardLeaderReplica with journal batch size 1 09:17:39,45 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:39,45 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:39,45 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:39,45 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#1753949713] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#573266883] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#291904566 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#-1512163513] 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1103416940 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,45 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/shard-member-3:snapshot-read#133220870] 09:17:39,47 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:39,47 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:39,48 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:39,49 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,49 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:17:39,49 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:39,49 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:17:39,49 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:39,49 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:39,50 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:39,50 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:17:39,50 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:17:39,50 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,50 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,55 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,55 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:39,55 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,55 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:17:39,55 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:17:39,55 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:17:39,234 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:39,235 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:39,235 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:39,274 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:39,274 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,304 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:17:39,304 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Received ElectionTimeout - switching to Candidate 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,304 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testRemoveShardLeaderReplica is no longer the leader 09:17:39,304 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 09:17:39,304 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:39,304 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,305 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:17:39,305 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:39,305 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 09:17:39,305 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:39,305 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is no longer the leader 09:17:39,305 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:39,305 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 09:17:39,604 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,604 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,604 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:17:39,614 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,614 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,614 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:17:39,684 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,684 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,685 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:17:39,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,714 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=2, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,714 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:17:39,865 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:39,865 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:39,866 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,872 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:39,936 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,954 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:17:39,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:39,956 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:39,956 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:39,956 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:39,957 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:39,957 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica, member-3-shard-cars-config_testRemoveShardLeaderReplica] 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,957 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:39,958 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:39,958 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:39,958 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:39,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 09:17:39,958 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:39,958 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:39,959 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:17:39,959 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:39,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,958 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:39,961 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,961 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:39,961 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,961 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:17:39,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:39,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,974 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:17:39,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,974 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:39,976 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:39,976 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,976 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 09:17:39,977 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 09:17:39,977 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,977 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:39,977 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 09:17:39,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:39,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:39,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:39,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,16 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:17:40,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:40,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,54 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:17:40,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,56 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:40,57 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:40,58 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:40,58 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,58 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 09:17:40,58 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:17:40,59 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 09:17:40,59 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,59 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:17:40,59 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:40,59 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,59 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-oper_testRemoveShardLeaderReplica, leaderId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,60 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:17:40,60 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 09:17:40,60 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-oper_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:40,60 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Received ElectionTimeout 09:17:40,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:40,84 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:17:40,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:40,84 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:40,85 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:40,85 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=null 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Set currentTerm=3, votedFor=member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:40,86 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:40,87 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,87 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 09:17:40,87 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#22260378] 09:17:40,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:40,87 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,88 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,88 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] 09:17:40,88 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 09:17:40,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] 09:17:40,88 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testRemoveShardLeaderReplica, leaderId=member-1-shard-people-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,88 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1009444241] 09:17:40,88 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 09:17:40,88 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:40,88 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:40,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testRemoveShardLeaderReplica updated: matchIndex: -1, nextIndex: 0 09:17:40,88 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:40,89 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1096350183] 09:17:40,89 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:40,89 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-2064519954] 09:17:40,89 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-2064519954] 09:17:40,89 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard cars memberName member-1, datastoreType Config 09:17:40,89 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 09:17:40,89 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:17:40,89 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@508102e1] 09:17:40,89 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onRemoveServer: RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'}, state: Idle 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]] 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=57a96602-f507-4622-a82f-634ef0067341}, logIndex: 0, isSendImmediate: true 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: Returning OK for operation RemoveServer{serverId='member-1-shard-cars-config_testRemoveShardLeaderReplica'} 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 09:17:40,90 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 09:17:40,90 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully removed replica for shard cars 09:17:40,91 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,91 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1009444241] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:40,91 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,91 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-3-shard-cars-config_testRemoveShardLeaderReplica, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 09:17:40,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:40,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 09:17:40,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:40,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 09:17:40,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,92 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Setting last applied to 0 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,92 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] with 10000 ms timeout 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 09:17:40,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testRemoveShardLeaderReplica, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica, votingState=VOTING]] 09:17:40,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 09:17:40,93 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 saved ShardManager snapshot successfully. Deleting the prev snapshot if available 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@96bbeb0 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@96bbeb0 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 09:17:40,93 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Successfully deleted prior snapshots 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,93 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testRemoveShardLeaderReplica: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,94 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]]]] 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Number of entries to be appended = 1 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]]] is present: existingEntryTerm: 3 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): After cleanup, lastIndex: 0, entries to be added from: 1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Log size is now 1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,95 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 09:17:40,95 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,95 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:17:40,95 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 09:17:40,95 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Commit index set to 0 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Setting last applied to 0 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Moving last applied index from -1 to 0 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]] 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Persisting ApplyJournalEntries with index=0 09:17:40,96 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,96 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:40,97 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:40,97 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:40,97 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 09:17:40,97 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 09:17:40,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,97 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Cluster state: [] 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 3 - updating term 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=null 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=4, votedFor=member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:40,98 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:40,99 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,99 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,99 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 09:17:40,100 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,100 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:17:40,144 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,144 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1009444241] 09:17:40,144 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:40,144 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,144 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-1069336654] 09:17:40,145 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1096350183] 09:17:40,145 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:40,145 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-2064519954] 09:17:40,145 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-2064519954] 09:17:40,145 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#22260378] 09:17:40,145 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:40,145 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 09:17:40,145 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:40,146 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,146 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:40,146 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onShutDown 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 09:17:40,146 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-config_testRemoveShardLeaderReplica 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70a3b5bf 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardDataTree - member-3-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70a3b5bf 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 09:17:40,146 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,146 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:17:40,147 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,147 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,148 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-config_testRemoveShardLeaderReplica, matchIndex: 0, lastIndex: 0, isVoting: true 09:17:40,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Follower's log matches - sending ElectionTimeout 09:17:40,148 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,149 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leader transfer complete 09:17:40,149 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Received TimeoutNow - switching to Candidate 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 5 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] to peer member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:40,150 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:40,150 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 09:17:40,150 AM [cluster-test-pekko.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:17:40,150 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cluster state: [] 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3]" message is greater than leader's term 4 - switching to Follower 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer in progress - processing RequestVote 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 3 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:40,151 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:40,151 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 09:17:40,151 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 09:17:40,151 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-config_testRemoveShardLeaderReplica] 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:17:40,152 AM [cluster-test-pekko.actor.default-dispatcher-35] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 09:17:40,152 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,153 AM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader changed to member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,153 AM [cluster-test-shard-dispatcher-36] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to member-2-shard-cars-config_testRemoveShardLeaderReplica in 6.973 ms 09:17:40,153 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testRemoveShardLeaderReplica, leaderId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderPayloadVersion=13] 09:17:40,153 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 09:17:40,153 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:17:40,153 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:17:40,153 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,153 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testRemoveShardLeaderReplica updated: matchIndex: 0, nextIndex: 1 09:17:40,153 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:40,153 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,154 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:40,154 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:40,154 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,154 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:40,154 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:40,154 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 09:17:40,154 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 09:17:40,154 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 09:17:40,155 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:17:40,155 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:40,155 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:40,156 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:40,156 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:40,160 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:40,160 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:40,160 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:40,160 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:40,161 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:40,176 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:40,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,176 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:40,176 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:17:40,176 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onShutDown 09:17:40,176 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onShutDown 09:17:40,176 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Initiating leader transfer 09:17:40,176 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-people-config_testRemoveShardLeaderReplica 09:17:40,176 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=13] 09:17:40,176 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d0f21ea 09:17:40,177 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d0f21ea 09:17:40,177 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 09:17:40,177 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Attempting to transfer leadership 09:17:40,177 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testRemoveShardLeaderReplica: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:40,296 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 09:17:40,297 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:40,375 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 09:17:40,376 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 09:17:40,595 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 5 09:17:40,596 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "AppendEntriesReply [term=5, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5]" message is greater than leader's term 3 - switching to Follower 09:17:40,596 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Set currentTerm=5, votedFor=null 09:17:40,596 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:17:40,596 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:17:40,596 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testRemoveShardLeaderReplica is no longer the leader 09:17:40,596 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 09:17:40,596 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testRemoveShardLeaderReplica, leaderId=null, leaderPayloadVersion=-1] 09:17:40,596 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:40,596 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 09:17:40,616 AM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:40,696 AM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:40,885 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:40,936 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:41,17 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:41,172 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3543878942082743549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:41,174 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3543878942082743549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:41,255 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:41,336 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:41,577 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:41,657 AM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:41,896 AM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:41,976 AM [cluster-test-shard-dispatcher-22] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:42,114 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 09:17:42,114 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.021 s 09:17:42,114 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer succeeded - sending PoisonPill 09:17:42,114 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 09:17:42,115 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-2038788481] 09:17:42,217 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:42,296 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:42,537 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:42,617 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:42,856 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:42,936 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:43,176 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:43,256 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:43,496 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:43,576 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:43,816 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:43,896 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:44,136 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:44,216 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:44,458 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:44,537 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:44,777 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:44,857 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:45,95 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:17:45,95 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:45,97 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 09:17:45,97 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:45,175 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:17:45,175 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:45,177 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 09:17:45,177 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:17:45,204 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,204 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-config_testRemoveShardLeaderReplica, member-2-shard-people-config_testRemoveShardLeaderReplica] 09:17:45,205 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,205 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-config_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:45,205 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:45,205 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:45,205 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:17:45,205 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:45,417 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:45,497 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:45,737 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:45,816 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:45,985 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:17:45,985 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:45,986 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:45,986 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:46,43 AM [cluster-test-pekko.actor.default-dispatcher-34] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:17:46,43 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:46,43 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:46,56 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:46,136 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:46,377 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:46,455 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:46,697 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:46,776 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:47,19 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:17:47,96 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:47,337 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:17:47,416 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:47,657 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:47,661 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:17:47,737 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:47,818 AM [cluster-test-pekko.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:17:47,977 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:48,56 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:17:48,297 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:48,376 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:48,617 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:48,697 AM [cluster-test-shard-dispatcher-12] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:48,937 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:49,16 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:49,256 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:17:49,336 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:17:49,412 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 09:17:49,412 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.236 s 09:17:49,412 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 09:17:49,412 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer failed - sending PoisonPill 09:17:49,412 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Leadership transfer expired 09:17:49,413 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 09:17:49,413 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:49,413 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:49,414 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:49,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:49,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:49,414 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:49,414 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 09:17:49,414 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 09:17:49,414 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:17:49,415 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 09:17:49,415 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:49,415 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:49,417 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:49,417 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:50,214 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:50,214 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-people-oper_testRemoveShardLeaderReplica, member-2-shard-people-oper_testRemoveShardLeaderReplica] 09:17:50,214 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:50,215 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-people-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:50,215 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:50,215 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,215 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:17:50,215 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [WARN] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Election: Leader has following peers: [member-3-shard-cars-oper_testRemoveShardLeaderReplica, member-2-shard-cars-oper_testRemoveShardLeaderReplica] 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-3-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-oper_testRemoveShardLeaderReplica: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:17:50,294 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,294 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:17:50,294 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:17:50,434 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4645474035699533924] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:50,435 AM [cluster-test-pekko.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:50,435 AM [cluster-test-pekko.actor.default-dispatcher-34] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [11/03/2024 09:17:50.439] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:50,463 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:17:50,464 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:50,464 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:17:50,464 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onShutDown 09:17:50,464 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Stopping Shard member-1-shard-people-config_testRemoveShardLeaderReplica 09:17:50,465 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:50,465 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:50,465 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:17:50,465 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:50,465 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:50,465 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:50,465 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onShutDown 09:17:50,465 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onShutDown 09:17:50,466 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 09:17:50,466 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:17:50,466 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:50,466 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:50,468 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:50,468 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [11/03/2024 09:17:50.473] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:50,497 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:17:50,503 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1396479570072344912] 09:17:50,503 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:50,505 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:50,505 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:50,506 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:50,506 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:50,529 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,531 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,532 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:50,532 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:50,532 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:50,535 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,541 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#354306240] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,541 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:50,543 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:50,544 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,544 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,544 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,545 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,545 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,594 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:17:50,598 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 09:17:50,598 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@2794f433} 09:17:50,602 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,602 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,602 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,605 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,605 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 09:17:50,605 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,605 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1183942910] 09:17:50,605 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,606 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,606 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1183942910] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,608 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 09:17:50,608 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,608 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,608 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#115007785] 09:17:50,608 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 09:17:50,608 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,609 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 09:17:50,609 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1987836256] 09:17:50,609 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1165625682 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,609 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1842497853 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,609 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-752603431] 09:17:50,609 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#115007785] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,610 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-1987836256] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,610 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1165625682] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,610 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1842497853] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,610 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-752603431] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,610 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#1390191688] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,611 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,611 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:50,611 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:50,611 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:17:50,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,611 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,612 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 09:17:50,613 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:50,613 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:17:50,613 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:50,613 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,613 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:50,613 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:17:50,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 09:17:50,613 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,614 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,614 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,614 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:50,614 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,614 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:17:50,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 09:17:50,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 09:17:50,614 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:50,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:17:50,614 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:17:50,614 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:17:50,614 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 09:17:50,650 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:17:50,652 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 09:17:50,652 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@4a93e814} 09:17:50,652 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:17:50,652 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 09:17:50,652 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,652 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1228367086] 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,653 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-263145847] 09:17:50,654 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 09:17:50,655 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 09:17:50,655 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, persistent : true 09:17:50,655 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#2036869237 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,656 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-103775316] 09:17:50,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-263145847] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1228367086] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,656 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with journal batch size 1 09:17:50,656 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-1=false} 09:17:50,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/shard-member-1:snapshot-read#-193969669] 09:17:50,656 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 09:17:50,657 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:50,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,657 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,657 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,658 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]], state: Idle 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:50,658 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Returning INVALID_REQUEST for operation ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] 09:17:50,658 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:17:50,658 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:17:50,659 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:17:50,659 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,659 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1978866179 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 09:17:50,659 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:17:50,659 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,659 AM [cluster-test-pekko.actor.default-dispatcher-23] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard cars java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$11.onComplete(ShardManager.java:1564) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:17:50,660 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#354306240] 09:17:50,660 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:50,660 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1183942910] 09:17:50,659 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,660 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#1183942910] 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower): Received TimeoutNow - switching to Candidate 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Election: Candidate has following voting peers: [] 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,660 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:50,660 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:50,660 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is no longer the leader 09:17:50,660 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:17:50,660 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Received ElectionTimeout 09:17:50,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Leader): Election: Leader has following peers: [] 09:17:50,661 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:17:50,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:50,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:50,661 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 09:17:50,661 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@704b8623 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 09:17:50,661 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@704b8623 09:17:50,661 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@12513fd5 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@12513fd5 09:17:50,661 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,661 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,662 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:50,662 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:17:50,662 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:50,662 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 09:17:50,662 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@394f407 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderId=member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard, leaderPayloadVersion=13] 09:17:50,662 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@394f407 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 09:17:50,662 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,663 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onShutDown 09:17:50,663 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@794df2e2 09:17:50,663 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@794df2e2 09:17:50,663 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:17:50,663 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:50,663 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:50,664 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:50,664 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:50,666 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:50,678 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:50,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4564121727801183785] 09:17:50,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 09:17:50,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:17:50,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:17:50,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:50,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:50,707 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:17:50,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:17:50,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:17:50,709 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,711 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,714 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:17:50,714 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1798793421] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-config-member-1 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,717 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,753 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,754 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@71ab69f9} 09:17:50,754 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,755 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,757 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,760 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,760 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,761 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-394258787] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,761 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] 09:17:50,761 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,761 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Shard created : member-1-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,761 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-2048633551] 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-763308941 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,762 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Shard created : member-1-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-2048633551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-763308941] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,762 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-473976931] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,762 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Starting recovery for member-1-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,763 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1180463136 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,763 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#159354323] 09:17:50,763 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1180463136] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,763 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#159354323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,763 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-1 09:17:50,763 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Starting recovery for member-1-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,763 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,764 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,764 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-473976931] 09:17:50,764 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,764 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,764 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,766 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,766 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,766 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:17:50,766 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,766 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:17:50,766 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,766 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,766 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,766 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,766 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,766 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,766 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,767 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,794 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,795 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@13c47c1f} 09:17:50,796 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,796 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,796 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-5777601] 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-650404603] 09:17:50,798 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Shard created : member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,798 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-5777601] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,798 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-96136640 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,798 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-615524943] 09:17:50,799 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-96136640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,799 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - Starting recovery for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,800 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Shard created : member-1-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,800 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-1316429884 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,800 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-1:snapshot-read#-1823294728] 09:17:50,800 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Starting recovery for member-1-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:17:50,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,801 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,802 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,802 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:17:50,802 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,802 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,802 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,802 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,804 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:50,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [8758347908844851955] 09:17:50,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 09:17:50,816 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:50,817 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:17:50,818 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:50,818 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:50,827 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,829 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,836 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,841 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1535324779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,841 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:17:50,850 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-config-member-2 09:17:50,850 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,850 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,859 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:17:50,863 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:50,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,871 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,872 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@2de62b00} 09:17:50,872 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-2-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#1112991375] 09:17:50,874 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Shard created : member-2-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#1112991375] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-934450147 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#132175197] 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-934450147] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,874 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#132175197] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,874 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Starting recovery for member-2-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,874 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-2-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,875 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,875 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#1498609281] 09:17:50,875 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1615742618 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,875 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1615742618] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,875 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#1498609281] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,875 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-2-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,876 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,876 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,877 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,878 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:17:50,878 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,878 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#832495773] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,879 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:17:50,879 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,879 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,879 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,879 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,879 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,879 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,879 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,879 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:17:50,879 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,880 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,880 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-2 09:17:50,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,881 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,881 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,881 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,881 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,881 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,881 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,881 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,881 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,881 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,882 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,883 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,884 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@44bb4b39} 09:17:50,884 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,884 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,884 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1152622676] 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1152622676] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,885 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-2-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,885 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#54060590] 09:17:50,886 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Shard created : member-2-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,886 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-183941785 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,886 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#54060590] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,886 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-928742052 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,886 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#1604956927] 09:17:50,886 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-2:snapshot-read#2027249970] 09:17:50,886 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - Starting recovery for member-2-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,886 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,887 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,887 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,887 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,887 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,887 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,887 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:17:50,887 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,888 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,888 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,888 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,888 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,888 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:17:50,901 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3279630054503999904] 09:17:50,901 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 09:17:50,904 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:50,904 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:17:50,904 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:17:50,904 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:17:50,919 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,920 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,923 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,933 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:17:50,934 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1016856554] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,936 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:17:50,941 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-config-member-3 09:17:50,941 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,942 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,944 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:17:50,945 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,945 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,945 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,948 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,949 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@bce1ebf} 09:17:50,949 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,950 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] 09:17:50,950 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,950 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-cars-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,951 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,951 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,951 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-3-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-130008521] 09:17:50,951 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Shard created : member-3-shard-people-config_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,951 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-cars-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,952 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:17:50,952 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - Starting recovery for member-3-shard-people-config_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,952 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1476306816 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,952 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#441222091] 09:17:50,952 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,952 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#765959933 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,953 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#225666155] 09:17:50,954 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,955 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-130008521] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1476306816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#441222091] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#765959933] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#225666155] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-12861997] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,955 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,955 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,956 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:17:50,956 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:17:50,956 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,959 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,959 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-oper-member-3 09:17:50,959 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,959 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,960 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:17:50,960 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,960 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,974 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,974 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,974 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,974 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,974 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:50,974 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:17:50,974 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,986 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,987 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f38f0766468@53229471} 09:17:50,987 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:17:50,988 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:50,988 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:17:50,988 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-706260126] 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Shard created : member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-706260126] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#903393262] 09:17:50,993 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Shard created : member-3-shard-people-oper_testChangeMemberVotingStatusForShard, persistent : true 09:17:50,994 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Starting recovery for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,994 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-1951279892] 09:17:50,994 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1888264137 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,994 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1448555452 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,994 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#903393262] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:17:50,994 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/shard-member-3:snapshot-read#-2020970197] 09:17:50,995 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Starting recovery for member-3-shard-people-oper_testChangeMemberVotingStatusForShard with journal batch size 1 09:17:50,995 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,995 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,995 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:17:50,995 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,995 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,995 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,995 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:50,995 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,996 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,996 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onReceiveRecover: Received message class org.apache.pekko.persistence.RecoveryCompleted$ from Actor[pekko://cluster-test/deadLetters] 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:17:50,996 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:17:50,996 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:50,996 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:50,996 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:17:50,996 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:51,86 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:17:51,86 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is no longer the leader 09:17:51,145 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:17:51,145 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-oper_testChangeMemberVotingStatusForShard, member-2-shard-cars-oper_testChangeMemberVotingStatusForShard] 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:17:51,165 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:51,166 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:51,166 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:51,166 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:17:51,166 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(pekko://cluster-test@127.0.0.1:2557, Joining), Member(pekko://cluster-test@127.0.0.1:2558, Up), Member(pekko://cluster-test@127.0.0.1:2559, Joining)] self member: Member(pekko://cluster-test@127.0.0.1:2558, Up) 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Received ElectionTimeout - switching to Candidate 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-oper_testChangeMemberVotingStatusForShard, member-3-shard-people-oper_testChangeMemberVotingStatusForShard] 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=1, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is no longer the leader 09:17:51,195 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:17:51,195 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 09:17:51,425 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,425 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,425 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:17:51,475 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,475 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,475 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:17:51,496 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,496 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,496 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:17:51,616 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,616 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=2, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,616 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:17:51,707 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:17:51,707 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,707 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,708 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,715 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,715 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,718 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,776 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:17:51,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,778 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:51,778 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:51,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:51,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:51,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-2-shard-cars-config_testChangeMemberVotingStatusForShard, member-3-shard-cars-config_testChangeMemberVotingStatusForShard] 09:17:51,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,779 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,780 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:51,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:51,780 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:17:51,780 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:17:51,780 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,780 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 09:17:51,781 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 09:17:51,781 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,781 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 09:17:51,781 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,781 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 09:17:51,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 09:17:51,782 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,826 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,826 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,826 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:17:51,826 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,826 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Cluster state: [] 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=null 09:17:51,827 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,827 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Election: Leader has following peers: [member-3-shard-people-config_testChangeMemberVotingStatusForShard, member-2-shard-people-config_testChangeMemberVotingStatusForShard] 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:17:51,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:17:51,830 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:17:51,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 09:17:51,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:17:51,830 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 09:17:51,830 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:51,830 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,831 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:17:51,831 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 09:17:51,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 09:17:51,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,834 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 09:17:51,836 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderPayloadVersion=13] 09:17:51,836 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:17:51,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-config_testChangeMemberVotingStatusForShard updated: matchIndex: -1, nextIndex: 0 09:17:51,836 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 09:17:51,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:51,836 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1798793421] 09:17:51,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:51,836 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] 09:17:51,836 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] 09:17:51,837 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1535324779] 09:17:51,837 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:51,837 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] 09:17:51,837 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] 09:17:51,837 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1016856554] 09:17:51,837 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:51,837 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] 09:17:51,837 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] 09:17:51,838 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard cars: {member-2=false, member-3=false} 09:17:51,838 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:17:51,838 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:51,839 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,839 AM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 09:17:51,839 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: getLeaderAddress leaderId = member-1-shard-cars-config_testChangeMemberVotingStatusForShard peerAddress = pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,839 AM [cluster-test-shard-dispatcher-36] [DEBUG] RaftActorServerConfigurationSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Not leader - forwarding to leader ActorSelection[Anchor(pekko://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard)] 09:17:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onChangeServersVotingStatus: ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]], state: Idle 09:17:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - Initiating ChangeServersVotingStatusState 09:17:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: true, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 09:17:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: New server configuration : [ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]] 09:17:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Replicate message: identifier: ServerOperationContextIdentifier{value=12b927d2-557d-4480-9dc2-d6f8a39a7e88}, logIndex: 0, isSendImmediate: true 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Setting last applied to 0 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Sending AppendEntries to follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Returning OK for operation ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorServerConfigurationSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] has been successfully replicated to a majority of followers 09:17:51,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 09:17:51,845 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,845 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:17:51,845 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard cars 09:17:51,845 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 09:17:51,845 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 09:17:51,845 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:51,845 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 09:17:51,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,845 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]]]] 09:17:51,845 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,845 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Number of entries to be appended = 1 09:17:51,845 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Append entry to log ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, address=null, votingState=NON_VOTING], PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING]] 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Updated server config: isVoting: false, peers: [PeerInfo [id=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=VOTING], PeerInfo [id=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, address=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard, votingState=NON_VOTING]] 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Log size is now 1 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Commit index set to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Setting last applied to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Moving last applied index from -1 to 0 09:17:51,846 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 09:17:51,846 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Applying state for log index 0 data ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]] 09:17:51,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 09:17:51,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:17:51,847 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,847 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,847 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=ServerConfigurationPayload [serverConfig=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]]] 09:17:51,847 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 09:17:51,847 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 09:17:51,847 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-config_testChangeMemberVotingStatusForShard updated: matchIndex: 0, nextIndex: 1 09:17:51,847 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:51,848 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Persisting ApplyJournalEntries with index=0 09:17:51,848 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 09:17:51,848 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1798793421] 09:17:51,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:17:51,849 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] 09:17:51,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-657523512] 09:17:51,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1535324779] 09:17:51,850 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:17:51,850 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] 09:17:51,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#-260754837] 09:17:51,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1016856554] 09:17:51,850 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:17:51,850 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] 09:17:51,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#1663347872] 09:17:51,850 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:51,850 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,850 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,850 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:17:51,850 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,850 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,851 AM [cluster-test-shard-dispatcher-36] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,851 AM [cluster-test-shard-dispatcher-36] [INFO] Shard - Stopping Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,851 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:17:51,851 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:17:51,851 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:51,851 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,851 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,851 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,851 AM [cluster-test-shard-dispatcher-36] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,851 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,852 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:17:51,852 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - Stopping Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,852 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:17:51,852 AM [cluster-test-shard-dispatcher-36] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:17:51,852 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:51,853 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:51,856 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:51,856 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:51,856 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:51,857 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:51,858 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:17:51,869 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:51,870 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,870 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,870 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:17:51,870 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,870 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,871 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:51,871 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:17:51,871 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:17:51,871 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:17:51,871 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:51,872 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,872 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,872 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:17:51,872 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,872 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:51,872 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - Stopping Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,872 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - Stopping Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,872 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:17:51,872 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:17:51,873 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:51,873 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:51,886 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,886 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,886 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:17:51,886 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,886 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:51,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:51,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=3, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,945 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:17:51,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:51,946 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:52,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=4, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,235 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 4 09:17:52,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,236 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,336 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:52,336 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=4, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,336 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 4 09:17:52,336 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,336 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,655 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:52,655 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=5, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,656 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 5 09:17:52,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:52,715 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:52,715 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=5, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,716 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 5 09:17:52,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:52,726 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:17:52,866 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3279630054503999904] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:52,885 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [8758347908844851955] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:17:52,894 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:52,894 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:17:52,895 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:52.897] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:17:52,912 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onShutDown 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Initiating leader transfer 09:17:52,913 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@141fc115 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1db645b3 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@141fc115 09:17:52,913 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, leaderId=null, leaderPayloadVersion=13] 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1db645b3 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Attempting to transfer leadership 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:52,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-2-shard-people-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:17:52,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Checking sendAppendEntries for follower member-3-shard-cars-config_testChangeMemberVotingStatusForShard: active: false, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:17:53,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:53,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Set currentTerm=6, votedFor=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:53,55 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 6 09:17:53,56 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:53,56 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:53,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Received ElectionTimeout 09:17:53,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Set currentTerm=6, votedFor=member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:53,116 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 6 09:17:53,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:53,116 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:53,376 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 09:17:53,376 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 462.9 ms 09:17:53,376 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 09:17:53,376 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 09:17:53,376 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 09:17:53,377 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 512.9 ms 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer failed - sending PoisonPill 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Leader): Leadership transfer expired 09:17:53,426 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:17:53,427 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:17:53,427 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:17:53,427 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:17:53,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:53,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:53,428 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:17:53,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:53,428 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onShutDown 09:17:53,428 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - Stopping Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:17:53,428 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - Stopping Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:17:53,428 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:17:53,428 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:17:53,429 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:17:53,429 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:17:53,431 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [11/03/2024 09:17:53.433] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config